-- Logs begin at Mon 2022-08-01 14:35:55 UTC, end at Mon 2022-08-01 15:40:39 UTC. -- Aug 01 15:09:07.842211 dv-cinder-847132-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 01 15:09:07.850286 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 01 15:09:07.852692 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 1 15:09:07 2022] *** Aug 01 15:09:07.852692 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 01 15:09:07.852692 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 01 15:09:07.852692 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: nodename: dv-cinder-847132-3-iscsi Aug 01 15:09:07.852692 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: machine: x86_64 Aug 01 15:09:07.852692 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: clock source: unix Aug 01 15:09:07.852692 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: pcre jit disabled Aug 01 15:09:07.852692 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: detected number of CPU cores: 4 Aug 01 15:09:07.852692 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: current working directory: / Aug 01 15:09:07.852692 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: detected binary path: /usr/bin/uwsgi-core Aug 01 15:09:07.852692 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: your processes number limit is 47680 Aug 01 15:09:07.852692 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: your memory page size is 4096 bytes Aug 01 15:09:07.853465 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: detected max file descriptor number: 2048 Aug 01 15:09:07.853465 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: lock engine: pthread robust mutexes Aug 01 15:09:07.853465 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: thunder lock: enabled Aug 01 15:09:07.853465 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 01 15:09:07.853465 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 01 15:09:07.899971 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: Python main interpreter initialized at 0x55f51c3309c0 Aug 01 15:09:07.899971 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: python threads support enabled Aug 01 15:09:07.899971 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: your server socket listen backlog is limited to 100 connections Aug 01 15:09:07.899971 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: your mercy for graceful operations on workers is 90 seconds Aug 01 15:09:07.900518 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: mapped 403077 bytes (393 KB) for 2 cores Aug 01 15:09:07.900518 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: *** Operational MODE: preforking *** Aug 01 15:09:07.900722 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: *** uWSGI is running in multiple interpreter mode *** Aug 01 15:09:07.900722 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: spawned uWSGI master process (pid: 80278) Aug 01 15:09:07.901003 dv-cinder-847132-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 01 15:09:07.905814 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: spawned uWSGI worker 1 (pid: 80291, cores: 1) Aug 01 15:09:07.905814 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: spawned uWSGI worker 2 (pid: 80292, cores: 1) Aug 01 15:09:07.905814 dv-cinder-847132-3-iscsi devstack@keystone.service[80278]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 01 15:09:13.590558 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 15:09:13.590558 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: from cryptography.utils import int_from_bytes Aug 01 15:09:13.591512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 15:09:13.591512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: from cryptography.utils import int_from_bytes Aug 01 15:09:13.642083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 15:09:13.642293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 15:09:13.642439 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] command line args: [] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 15:09:13.642708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 15:09:13.642883 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ================================================================================ {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 15:09:13.643141 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] admin_token = **** {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.643385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] config_dir = [] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.643569 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.643753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] config_source = [] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.643930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] debug = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.644137 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.644517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.644698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.644948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.645201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.645429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] list_limit = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.645609 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] log_config_append = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.645826 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.646009 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] log_dir = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.646188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] log_file = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.646437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.646669 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.646914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.647162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.647335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.647519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.647752 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.647996 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.648228 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.648458 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.648688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.648919 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.649165 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.649409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.649683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.650210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.650397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] publish_errors = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.650635 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.650866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.651046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.651297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.651532 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.651767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] standard_threads = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.651999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.652246 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.652423 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.652605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] use_journal = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.652780 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] use_json = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.652956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] use_stderr = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.653130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] use_syslog = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.653362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.653619 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.653875 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.654136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.654376 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.654623 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.654859 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.655105 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.655342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.655590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.655958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.656215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.656456 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.656703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.656938 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.657186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.657432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.657668 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.657945 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.658202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.658450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.658686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.658939 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.659207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.659456 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.660007 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.660007 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.660254 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.660394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.660626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.660871 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.661117 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.661347 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.661698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.661974 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.662226 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.662465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.662724 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.662962 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.663189 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.663427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.663653 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.663879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.663962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 15:09:13.663962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: from cryptography.utils import int_from_bytes Aug 01 15:09:13.664477 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.664549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 15:09:13.664549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: from cryptography.utils import int_from_bytes Aug 01 15:09:13.664720 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.664720 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.664869 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.665014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.665248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.665478 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.665710 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.665968 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.666215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.666451 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.666688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.666921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.667156 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.667389 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.667623 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.667851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.668099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.668332 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.668567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.668798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.669056 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.669299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.669527 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.669761 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.670016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.670281 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.670513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.670759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.671001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.671236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.671580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.671818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.672065 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.672301 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.672536 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.672777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.673012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.673253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.673578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.673830 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.674067 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.674327 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.674566 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.674805 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.675040 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.675280 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.675515 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.675749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.675981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.676211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.676445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.677201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.677201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.677451 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.677574 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.677848 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.678104 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.678343 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.678603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.680503 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.680769 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.681018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] auth.external = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.681264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.681573 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.681838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.682080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] auth.password = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.682318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] auth.token = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.682563 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.682803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.683062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.683351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.683619 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.683782 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.684075 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.684341 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] credential.caching = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.684486 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.684835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.686140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.686140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.686140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.686140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.686140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.686140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.686944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.686944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.686944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.687186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.687405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.687649 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.687913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.688252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] federation.caching = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.688494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.688743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.688991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.689224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.689473 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.689713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.689991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.690230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.690484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.690724 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.690971 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.691207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity.caching = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.691457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.691703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.692037 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.692513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.693018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.693433 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.693889 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.694357 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.694680 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.694926 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.695166 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.695404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.695662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.695913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.696164 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.696425 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.696681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.696957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.697197 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.697435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.697667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.697928 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.698163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.698402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.698644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.698885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.699133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.699370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.699619 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.699869 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.700102 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.700344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.700596 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.700824 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.701063 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.701291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.701527 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.701763 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.702028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.702266 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.702504 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.702750 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.702997 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.703231 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.703461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.703710 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.703958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.704195 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.704430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.704666 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.704896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.705135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.705394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.705624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.705908 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.706247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.706499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.706733 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.706961 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.707197 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.707431 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.707666 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.707893 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.708139 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.708412 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.708658 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.708901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.709144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.709373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.709617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.709873 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.710110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.710344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.710575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.710819 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.711062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.711298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.711549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.711779 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.711926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 15:09:13.712092 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.712232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 15:09:13.712383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.712521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.712673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] command line args: [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 15:09:13.712811 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.712945 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.713207 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 15:09:13.713357 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.713496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ================================================================================ {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 15:09:13.713630 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.713630 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.713924 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] admin_token = **** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.714057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] resource.caching = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.714057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.714267 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] config_dir = [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.714383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.714532 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.714651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.714766 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] config_source = [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.714882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.714997 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.715108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] debug = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.715237 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.715392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.715554 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.715669 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.715820 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.715957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.716114 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] role.caching = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.716236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.716350 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] role.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.716555 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.716683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.716813 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.716986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.717123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.717282 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.717448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] list_limit = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.717576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.717691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] log_config_append = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.717838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.718036 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.718151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.718266 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.718382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] log_dir = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.718527 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.718658 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] log_file = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.718807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.718941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.719054 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.719187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.719394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.719532 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.719679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.719869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.720065 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.720281 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.720482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.720632 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.720767 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.720901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.721037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.721172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.721302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.721447 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.721582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.721714 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.721874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.722010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.722147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.722306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.722441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.722572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.722706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.722852 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.722983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.723131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.723318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.723456 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.723589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.723788 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.723931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.724145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.724284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.724538 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.724711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.724953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] token.caching = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.725380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.725534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.48/identity {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.725702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] publish_errors = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.725874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.726123 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.726270 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.726505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.726739 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.726884 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.727154 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.727311 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.727543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.727686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.727928 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] standard_threads = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.728100 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.728326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.728483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.728725 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.728933 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.729075 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.729307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] use_journal = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.729449 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.729627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] use_json = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.729769 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.730001 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] use_stderr = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.730135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.730268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.730402 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] use_syslog = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.730557 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.730689 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 15:09:13.730834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.730967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.731098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.731231 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.731481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.connection = **** {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.731632 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.731786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.731972 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.732115 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.732249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.732387 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.732522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.732674 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.732837 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.732978 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.733112 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.733242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.733360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.733574 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.733712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.733900 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.734110 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.734256 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.734391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.734552 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.735042 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.735299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.735444 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.735704 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.735851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.736098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.736242 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.736498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.736644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.736887 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.737095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.737319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.737464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80292) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 15:09:13.737721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.738117 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.738449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.738814 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.739158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.739491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.739864 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.740229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.740576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.740931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.741260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.741588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.741965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.742299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.742641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.742997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.743322 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.743654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.743996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.744482 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.744749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.745103 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.745428 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.745759 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.746211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.746569 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.746897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.747232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.747560 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.747894 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.748270 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.748598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.748927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.749254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.749671 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.751630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.751630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.751630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.751630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.751630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.751630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.751630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.751630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.751630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.757744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.757744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.757744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.757744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.757744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.757744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.757744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.757744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.757744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.761621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.761621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.761621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.761621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.761621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.761621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.761621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.761621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.761621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.762695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.762695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.762695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.762695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.762695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.762695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.762695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.762695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.762695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.763849 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.763849 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.764074 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.764309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.764585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.764830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.765069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.765069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.765507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.765757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.766029 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.766275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.766514 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.766770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.767010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.767270 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.767533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.767780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.768011 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] auth.external = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.768243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.768542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.768774 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.769006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] auth.password = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.769236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] auth.token = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.769477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.769711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.769987 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.770220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.770467 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.770718 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.770995 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.771233 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] credential.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.771480 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.771730 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.771976 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.772215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.772460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.772693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.772959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.773211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.773448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.773706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.774024 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.774271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.774535 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.774780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.775040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.775278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] federation.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.775528 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.775777 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.776025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.776257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.776506 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.776745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.777000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.777238 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.777495 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.777734 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.778023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.778259 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.778507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.778756 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.779000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.779289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.779469 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.779703 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.779942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.780193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.780446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.780679 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.781014 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.781122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.781363 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.781609 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.781887 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.782147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.782393 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.782667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.782907 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.783143 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.783375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.783617 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.783848 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.784083 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.784327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.784565 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.784813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.785044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.785291 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.785537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.785773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.786043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.786291 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.786604 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.786845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.787075 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.787315 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.787552 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.787800 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.788038 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.788392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.788515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.788764 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.788996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.789227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.789476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.789722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.789985 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.790223 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.790458 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.790690 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.790932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.791193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.791426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.791672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.791920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.792163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.792400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.792630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.792867 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.793157 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.793339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.793576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.793844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.794092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.794337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.794581 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.794824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.795056 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.795306 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.795543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.795779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.796017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.796250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.796495 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.796743 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.796979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.797232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.797461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.797706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.798060 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.798183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.798425 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.798828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.799071 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.799304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.799543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.799786 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] resource.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.800034 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.800284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.800520 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.800770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.801021 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.801256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.801501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.801739 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.802006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.802283 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] role.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.802531 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] role.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.802763 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.803012 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.803258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.803504 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.803752 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.803995 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.804240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.804483 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.804730 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.804961 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.805206 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.805451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.805696 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.805965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.806428 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.806673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.806927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.807178 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.807427 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.807679 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.807915 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.808173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.808427 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.808666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.808898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.809132 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.809362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.809613 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.809895 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.810145 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.810382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.810617 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.810847 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.811085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] token.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.811322 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.811569 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.811803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.812057 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.812302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.812538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.812870 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.813002 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.813244 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.813479 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.813719 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.813978 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.814223 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.814468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.814696 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.814937 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.815236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.815497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.connection = **** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.815744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.815991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.816228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.816468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.816703 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.816940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.817176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.817413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.817682 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.817988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.818235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.818472 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.818722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.818959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.819203 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.819431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.819671 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.819907 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 15:09:13.820076 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80291) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 15:09:13.850610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.850808 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.850961 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.859595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.864138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.864334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.864572 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.874007 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.890280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.890582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.891863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.892072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.944043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.945544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.994147 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.994377 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.995339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:13.995581 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.065335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.065549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.065804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.066001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.066249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.066465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.066784 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.067325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.067547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.067794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.067991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.090496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.090723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.090815 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.090893 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.090968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.091043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.091118 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.091202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.091279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.091343 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.091416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.091491 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.091573 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.091649 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.103048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.103302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.116721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.116934 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.117187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.117298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.117364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.117425 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.117496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.117571 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.117708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.117868 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.117953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.118019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80291) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.118175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.118385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80292) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 15:09:14.135076 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.135244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.135315 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 15:09:14.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 15:09:14.135622 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.135751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.135820 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 15:09:14.135903 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 15:09:14.136073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.136212 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.136282 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.136357 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.136571 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.136694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.136809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.136898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.146650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.146824 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.146900 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.147084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.147193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.147450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.147538 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.147730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.154459 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.154731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.157687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.157990 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.158273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.158665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.158946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.159207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.159297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.159505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.159636 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.159751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.159854 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.160053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.160180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.160339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.160466 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.160563 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.160643 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.160904 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.161176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.161476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.161738 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.162032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.183771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.185209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.185337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.185554 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.186759 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.187027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.190584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.190863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.191138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.191774 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.192063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.192345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.197698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.198004 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.199054 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.199341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.202250 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.202513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.202993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.203442 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.203782 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.203888 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.203959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.204158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.204252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.204458 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.204545 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.204731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.204846 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.205014 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.205284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.205560 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.221243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.221523 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.226999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.227246 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.227392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.230463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.230761 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.231266 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.235027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.235380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.235651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.238985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.239894 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.242274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.242557 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.242849 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.243121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.243355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.243493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.243731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.246374 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.246678 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.247144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.247521 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.247865 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.248273 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.261285 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.261592 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.261969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.262217 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.264734 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.265051 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.265322 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.265592 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.269323 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.271144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.271416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.273211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.274121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.274414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.274691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.274829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.275101 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.275188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.275385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.277952 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.278235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.278534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.278809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.279093 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.287074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.287380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.287665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.288198 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.288569 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.288878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.289158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.290871 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.291174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.291455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.291733 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.292011 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.292289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.292563 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.305933 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.308936 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.310219 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.311932 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.313226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.313408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.315048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.315324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.315601 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.316225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.318966 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.320513 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.320795 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.321073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.322444 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.323956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.327021 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.327308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.327594 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.327691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.327876 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.329208 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.332183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.332470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.332752 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.333029 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.337428 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.337921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.338370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.342468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.342763 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.343063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.345822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80292) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.346209 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.346486 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.346767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.347059 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.347324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.347690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.348152 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.348629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.349151 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.349591 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.349890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.350234 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80291) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 15:09:14.350402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.350518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.350776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.350927 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80292) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.351048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.351344 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.351621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.351893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.352168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.352549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.352831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.353159 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.353388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.353660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.353940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80291) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 15:09:14.389166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55f51c3309c0 pid: 80291 (default app) Aug 01 15:09:14.390814 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55f51c3309c0 pid: 80292 (default app) Aug 01 15:09:14.392742 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b1eba-d28a-44fc-ae6e-0ca29ded155a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:14.393030 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b1eba-d28a-44fc-ae6e-0ca29ded155a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:14.393307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b1eba-d28a-44fc-ae6e-0ca29ded155a None None] PATH_INFO: `/v3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:14.397743 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1/1] 10.222.0.48 () {50 vars in 791 bytes} [Mon Aug 1 15:09:14 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:22.240349 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c355b9c-2488-47ab-a125-7cb123909895 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:22.240892 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c355b9c-2488-47ab-a125-7cb123909895 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:22.240892 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c355b9c-2488-47ab-a125-7cb123909895 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:22.245136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1/2] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:22.252069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38893ef4-434f-4242-8916-2c4de7aed6d5 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:22.252979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38893ef4-434f-4242-8916-2c4de7aed6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:22.252979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38893ef4-434f-4242-8916-2c4de7aed6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:22.326651 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: INFO dbcounter [None req-38893ef4-434f-4242-8916-2c4de7aed6d5 None None] Registered counter for database keystone Aug 01 15:09:22.347538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG oslo_db.sqlalchemy.engines [None req-38893ef4-434f-4242-8916-2c4de7aed6d5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80291) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 15:09:22.354426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writer thread running {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 15:09:22.547438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG passlib.handlers.bcrypt [None req-38893ef4-434f-4242-8916-2c4de7aed6d5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80291) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 15:09:22.547850 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG passlib.handlers.bcrypt [None req-38893ef4-434f-4242-8916-2c4de7aed6d5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80291) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 15:09:22.603695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-38893ef4-434f-4242-8916-2c4de7aed6d5 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:22.605373 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-38893ef4-434f-4242-8916-2c4de7aed6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:22.752789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 2/3] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 15:09:22.898753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c355b9c-2488-47ab-a125-7cb123909895 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:22.979852 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: INFO dbcounter [None req-8c355b9c-2488-47ab-a125-7cb123909895 None None] Registered counter for database keystone Aug 01 15:09:23.002510 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG oslo_db.sqlalchemy.engines [None req-8c355b9c-2488-47ab-a125-7cb123909895 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80292) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 15:09:23.009760 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writer thread running {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 15:09:23.131653 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c355b9c-2488-47ab-a125-7cb123909895 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:23.136453 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a3196-37e0-4665-9f68-fdc4cbff793f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:23.137130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a3196-37e0-4665-9f68-fdc4cbff793f admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:23.137393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a3196-37e0-4665-9f68-fdc4cbff793f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:23.137662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a3196-37e0-4665-9f68-fdc4cbff793f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:23.146737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a3196-37e0-4665-9f68-fdc4cbff793f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:23.198333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: WARNING keystone.common.rbac_enforcer.enforcer [None req-026a3196-37e0-4665-9f68-fdc4cbff793f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 15:09:23.198333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a3196-37e0-4665-9f68-fdc4cbff793f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:23.209085 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 2/4] 10.222.0.48 () {58 vars in 1324 bytes} [Mon Aug 1 15:09:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:09:23.215253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38893ef4-434f-4242-8916-2c4de7aed6d5 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:23.231185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38893ef4-434f-4242-8916-2c4de7aed6d5 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:23.236899 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bfbdbd-1cfb-4699-8a8c-deb0396a2669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:23.238115 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfbdbd-1cfb-4699-8a8c-deb0396a2669 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:23.238396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfbdbd-1cfb-4699-8a8c-deb0396a2669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:23.238668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bfbdbd-1cfb-4699-8a8c-deb0396a2669 admin admin] PATH_INFO: `/v3/projects/ef3b9d837c2248a7ae12fa99e632e776` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:23.240683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bfbdbd-1cfb-4699-8a8c-deb0396a2669 admin admin] RBAC: Authorizing `identity:get_project(project_id=ef3b9d837c2248a7ae12fa99e632e776)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:23.292475 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b5bfbdbd-1cfb-4699-8a8c-deb0396a2669 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 15:09:23.293205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bfbdbd-1cfb-4699-8a8c-deb0396a2669 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:23.297687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 3/5] 10.222.0.48 () {56 vars in 1207 bytes} [Mon Aug 1 15:09:23 2022] GET /identity/v3/projects/ef3b9d837c2248a7ae12fa99e632e776 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:25.273993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e90dec-8ff1-4fd3-ac22-1ceed330f02f None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:25.274519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e90dec-8ff1-4fd3-ac22-1ceed330f02f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:25.274519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e90dec-8ff1-4fd3-ac22-1ceed330f02f None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:25.278514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 3/6] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:09:25.284301 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3d9da-3afa-48fe-81b3-1dba23afeb39 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:25.284781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3d9da-3afa-48fe-81b3-1dba23afeb39 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:25.284874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3d9da-3afa-48fe-81b3-1dba23afeb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:25.305292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-bca3d9da-3afa-48fe-81b3-1dba23afeb39 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:25.306864 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bca3d9da-3afa-48fe-81b3-1dba23afeb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:25.315427 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 4/7] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 15:09:25.337970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e90dec-8ff1-4fd3-ac22-1ceed330f02f None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:25.352510 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e90dec-8ff1-4fd3-ac22-1ceed330f02f None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:25.357351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0484a-a061-4abe-8e54-979d812f74ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:25.358037 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0484a-a061-4abe-8e54-979d812f74ee admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:25.358324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0484a-a061-4abe-8e54-979d812f74ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:25.358607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0484a-a061-4abe-8e54-979d812f74ee admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:25.368139 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a0484a-a061-4abe-8e54-979d812f74ee admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:25.370292 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a0484a-a061-4abe-8e54-979d812f74ee admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:25.380086 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 4/8] 10.222.0.48 () {58 vars in 1324 bytes} [Mon Aug 1 15:09:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:09:25.386078 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3d9da-3afa-48fe-81b3-1dba23afeb39 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:25.395713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3d9da-3afa-48fe-81b3-1dba23afeb39 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:25.400228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de643146-2fe8-486b-b713-43fdb7580f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:25.400876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de643146-2fe8-486b-b713-43fdb7580f2d admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:25.401160 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de643146-2fe8-486b-b713-43fdb7580f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:25.401435 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de643146-2fe8-486b-b713-43fdb7580f2d admin admin] PATH_INFO: `/v3/users/1b4b6acdd88e421a86924fd2c155794f` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:25.402845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de643146-2fe8-486b-b713-43fdb7580f2d admin admin] RBAC: Authorizing `identity:get_user(user_id=1b4b6acdd88e421a86924fd2c155794f)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:25.404703 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de643146-2fe8-486b-b713-43fdb7580f2d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:25.406312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 5/9] 10.222.0.48 () {56 vars in 1198 bytes} [Mon Aug 1 15:09:25 2022] GET /identity/v3/users/1b4b6acdd88e421a86924fd2c155794f => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:27.512848 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e85efe-d525-4302-a519-f08934f7d2f0 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:27.513421 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e85efe-d525-4302-a519-f08934f7d2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:27.513421 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e85efe-d525-4302-a519-f08934f7d2f0 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:27.514754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 5/10] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:27.519978 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f7b57-db74-4bd3-ae2b-ce7337ebd694 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:27.520511 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f7b57-db74-4bd3-ae2b-ce7337ebd694 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:27.520670 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f7b57-db74-4bd3-ae2b-ce7337ebd694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:27.533144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6111340-5fc0-4150-b402-1528c5ea6f10 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:27.533470 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6111340-5fc0-4150-b402-1528c5ea6f10 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:27.534046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6111340-5fc0-4150-b402-1528c5ea6f10 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:27.535117 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 6/11] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:09:27.541066 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e01e0f-bc1d-4399-b237-764c9c9b726b None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:27.541355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e01e0f-bc1d-4399-b237-764c9c9b726b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:27.541632 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e01e0f-bc1d-4399-b237-764c9c9b726b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:27.541999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-0a7f7b57-db74-4bd3-ae2b-ce7337ebd694 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:27.543229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0a7f7b57-db74-4bd3-ae2b-ce7337ebd694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:27.585735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG passlib.handlers.bcrypt [None req-e2e01e0f-bc1d-4399-b237-764c9c9b726b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80292) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 15:09:27.586159 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG passlib.handlers.bcrypt [None req-e2e01e0f-bc1d-4399-b237-764c9c9b726b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80292) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 15:09:27.632036 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-e2e01e0f-bc1d-4399-b237-764c9c9b726b None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:27.633663 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e2e01e0f-bc1d-4399-b237-764c9c9b726b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:27.636232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 6/12] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:27.659256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f7b57-db74-4bd3-ae2b-ce7337ebd694 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:27.684274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:27.747359 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 7/13] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:09:27.770820 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e01e0f-bc1d-4399-b237-764c9c9b726b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:27.773970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f7b57-db74-4bd3-ae2b-ce7337ebd694 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:27.812158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43d4dc-331a-4ad0-b669-ef21e586407a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:27.812885 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43d4dc-331a-4ad0-b669-ef21e586407a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:27.813097 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43d4dc-331a-4ad0-b669-ef21e586407a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:27.813373 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f43d4dc-331a-4ad0-b669-ef21e586407a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:27.826195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f43d4dc-331a-4ad0-b669-ef21e586407a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:27.835378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-0f43d4dc-331a-4ad0-b669-ef21e586407a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:09:27.835378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:27.835378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:27.835378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:27.835378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:27.835378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:27.835378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:27.835378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:27.835378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:27.835378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:27.835378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:27.835378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:27.835378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:27.835378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:27.835378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:27.835378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:27.836549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:27.836549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:27.836549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:27.836549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:27.836549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:27.836549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:27.836549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:27.836549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:27.836549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:27.836549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:27.836549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:27.836549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:27.836549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:27.836549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:27.836549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:27.836549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:27.837572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:27.837572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:27.837572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:27.837572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:27.837572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:27.837572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:27.837572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:27.837572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:27.837572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:27.837572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:27.837572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:09:27.837572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:09:27.837572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 7/14] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:09:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:09:27.845927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43d4dc-331a-4ad0-b669-ef21e586407a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:27.889308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e01e0f-bc1d-4399-b237-764c9c9b726b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:27.891936 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:27.938601 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc4908-e9b6-4b60-b558-2584e2bbb14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:27.940350 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc4908-e9b6-4b60-b558-2584e2bbb14e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:27.940350 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc4908-e9b6-4b60-b558-2584e2bbb14e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:27.940803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc4908-e9b6-4b60-b558-2584e2bbb14e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:27.964242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:27.995997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f43d4dc-331a-4ad0-b669-ef21e586407a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:28.027677 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc4908-e9b6-4b60-b558-2584e2bbb14e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:28.029123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc4908-e9b6-4b60-b558-2584e2bbb14e None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:28.032659 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2ad0f-a9f0-42d1-9a8d-109b3d31b8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:28.033307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2ad0f-a9f0-42d1-9a8d-109b3d31b8ff None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:28.033590 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2ad0f-a9f0-42d1-9a8d-109b3d31b8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:28.033897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2ad0f-a9f0-42d1-9a8d-109b3d31b8ff None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:28.034646 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2ad0f-a9f0-42d1-9a8d-109b3d31b8ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:28.034949 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2ad0f-a9f0-42d1-9a8d-109b3d31b8ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:28.036497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2ad0f-a9f0-42d1-9a8d-109b3d31b8ff None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:28.049890 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 8/15] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:09:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:28.055589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2ad0f-a9f0-42d1-9a8d-109b3d31b8ff None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:28.144322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 8/16] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:09:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:09:28.150131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc4908-e9b6-4b60-b558-2584e2bbb14e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:28.153123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:28.176714 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2ad0f-a9f0-42d1-9a8d-109b3d31b8ff None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:28.279057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc4908-e9b6-4b60-b558-2584e2bbb14e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:28.314654 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6df8c4-55fb-41c7-9d1c-aec5e2714546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:28.315523 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6df8c4-55fb-41c7-9d1c-aec5e2714546 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:28.315796 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6df8c4-55fb-41c7-9d1c-aec5e2714546 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:28.316066 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6df8c4-55fb-41c7-9d1c-aec5e2714546 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:28.327545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be121aa8-d800-43df-9daa-90869991eab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:28.328211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be121aa8-d800-43df-9daa-90869991eab6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:28.328492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be121aa8-d800-43df-9daa-90869991eab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:28.328762 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be121aa8-d800-43df-9daa-90869991eab6 None admin] PATH_INFO: `/v3/users/1b4b6acdd88e421a86924fd2c155794f` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:28.330098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be121aa8-d800-43df-9daa-90869991eab6 None admin] RBAC: Authorizing `identity:get_user(user_id=1b4b6acdd88e421a86924fd2c155794f)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:28.331600 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be121aa8-d800-43df-9daa-90869991eab6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:28.333337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 9/17] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:28 2022] GET /identity/v3/users/1b4b6acdd88e421a86924fd2c155794f => generated 247 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:09:28.335031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6df8c4-55fb-41c7-9d1c-aec5e2714546 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:28.336515 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6df8c4-55fb-41c7-9d1c-aec5e2714546 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:28.339061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be121aa8-d800-43df-9daa-90869991eab6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:28.355230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-5f6df8c4-55fb-41c7-9d1c-aec5e2714546 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:09:28.355230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:28.355230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:28.355230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:28.355230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:28.355230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:28.355230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:28.355230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:28.355230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:28.355230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:28.355230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:28.355230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:28.355230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:09:28.355230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:09:28.355230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:28.356163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:28.357171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:09:28.357171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:09:28.357171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:09:28.357171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:09:28.357398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:28.357465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 9/18] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:09:28.362589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6df8c4-55fb-41c7-9d1c-aec5e2714546 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:28.421812 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:28.452392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be121aa8-d800-43df-9daa-90869991eab6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:28.468012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6df8c4-55fb-41c7-9d1c-aec5e2714546 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:28.488711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574b2d7-3e35-4d0e-8648-2e2c0833fefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:28.489421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574b2d7-3e35-4d0e-8648-2e2c0833fefa None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:28.489749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574b2d7-3e35-4d0e-8648-2e2c0833fefa None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:28.490056 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574b2d7-3e35-4d0e-8648-2e2c0833fefa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:28.491251 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574b2d7-3e35-4d0e-8648-2e2c0833fefa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:28.492727 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574b2d7-3e35-4d0e-8648-2e2c0833fefa None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:28.499156 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 10/19] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:09:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:28.500147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574b2d7-3e35-4d0e-8648-2e2c0833fefa None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:28.509671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b029d0-96e1-4ec5-aa72-141f821eb9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:28.510357 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b029d0-96e1-4ec5-aa72-141f821eb9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:28.510642 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b029d0-96e1-4ec5-aa72-141f821eb9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:28.510917 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b029d0-96e1-4ec5-aa72-141f821eb9e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:28.511634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b029d0-96e1-4ec5-aa72-141f821eb9e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:28.511943 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b029d0-96e1-4ec5-aa72-141f821eb9e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:28.513365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b029d0-96e1-4ec5-aa72-141f821eb9e6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:28.532753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 10/20] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:09:28.616666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:28.640047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574b2d7-3e35-4d0e-8648-2e2c0833fefa None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:28.678747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e7629-669f-41d5-b307-2c4ff8ca71d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:28.679393 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e7629-669f-41d5-b307-2c4ff8ca71d5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:28.679678 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e7629-669f-41d5-b307-2c4ff8ca71d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:28.679956 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e7629-669f-41d5-b307-2c4ff8ca71d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:28.680784 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e7629-669f-41d5-b307-2c4ff8ca71d5 None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.domain.id=default, user.id=1b4b6acdd88e421a86924fd2c155794f) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:28.681098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e7629-669f-41d5-b307-2c4ff8ca71d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:28.682632 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e7629-669f-41d5-b307-2c4ff8ca71d5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:28.690365 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 11/21] 10.222.0.48 () {56 vars in 1322 bytes} [Mon Aug 1 15:09:28 2022] GET /identity/v3/role_assignments?user.id=1b4b6acdd88e421a86924fd2c155794f&scope.domain.id=default&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 233 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:09:31.148002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cb231-bd9f-4608-99ea-7e7c9c71ea87 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:31.149107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cb231-bd9f-4608-99ea-7e7c9c71ea87 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:31.149607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7cb231-bd9f-4608-99ea-7e7c9c71ea87 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:31.151302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 11/22] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:31.159737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331bafb-c401-474c-9e4d-b25269603d4d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:31.161228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331bafb-c401-474c-9e4d-b25269603d4d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:31.162069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331bafb-c401-474c-9e4d-b25269603d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:31.190638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7331bafb-c401-474c-9e4d-b25269603d4d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:31.192269 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7331bafb-c401-474c-9e4d-b25269603d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:31.197762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5c3e3-6333-4f7b-9856-f1e096b8a2c6 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:31.198308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5c3e3-6333-4f7b-9856-f1e096b8a2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:31.198801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5c3e3-6333-4f7b-9856-f1e096b8a2c6 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:31.200351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 12/23] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:09:31.206793 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bd30e-7d17-4214-bb07-f83f9a1ae31d None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:31.207355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bd30e-7d17-4214-bb07-f83f9a1ae31d None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:31.207922 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bd30e-7d17-4214-bb07-f83f9a1ae31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:31.235873 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-a06bd30e-7d17-4214-bb07-f83f9a1ae31d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:31.237538 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-a06bd30e-7d17-4214-bb07-f83f9a1ae31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:31.344015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 12/24] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:31.384123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 13/25] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:31.391444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331bafb-c401-474c-9e4d-b25269603d4d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:31.399571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7c900-e1e1-4c05-b7a0-51c17d0a10e9 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:31.400187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7c900-e1e1-4c05-b7a0-51c17d0a10e9 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:31.400668 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7c900-e1e1-4c05-b7a0-51c17d0a10e9 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:31.402286 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 14/26] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:31.409772 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15edc87-1238-4574-afff-b9c1fd509386 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:31.410427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15edc87-1238-4574-afff-b9c1fd509386 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:31.410973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15edc87-1238-4574-afff-b9c1fd509386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:31.434156 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-e15edc87-1238-4574-afff-b9c1fd509386 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:31.435938 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e15edc87-1238-4574-afff-b9c1fd509386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:31.498241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:31.500040 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:31.528190 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331bafb-c401-474c-9e4d-b25269603d4d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:31.533508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 15/27] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:09:31.534720 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15edc87-1238-4574-afff-b9c1fd509386 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:31.572494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9fb85-c83a-4c1f-ab8a-6a30636195e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:31.573562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9fb85-c83a-4c1f-ab8a-6a30636195e4 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:31.573887 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9fb85-c83a-4c1f-ab8a-6a30636195e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:31.574169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9fb85-c83a-4c1f-ab8a-6a30636195e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:31.575182 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9fb85-c83a-4c1f-ab8a-6a30636195e4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:31.576913 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9fb85-c83a-4c1f-ab8a-6a30636195e4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:31.668128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15edc87-1238-4574-afff-b9c1fd509386 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:31.705558 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8698924-3e62-4ea4-a6bc-6cdbe58fcaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:31.706313 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8698924-3e62-4ea4-a6bc-6cdbe58fcaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:31.706596 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8698924-3e62-4ea4-a6bc-6cdbe58fcaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:31.706921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8698924-3e62-4ea4-a6bc-6cdbe58fcaa6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:31.713751 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 13/28] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 1 15:09:31 2022] POST /identity/v3/roles => generated 219 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:09:31.715700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27faf1e7-f941-4181-b992-b87bf92f2670 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:31.716240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27faf1e7-f941-4181-b992-b87bf92f2670 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:31.716730 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27faf1e7-f941-4181-b992-b87bf92f2670 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:31.718320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 14/29] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:31.719299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27faf1e7-f941-4181-b992-b87bf92f2670 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:31.720614 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8698924-3e62-4ea4-a6bc-6cdbe58fcaa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:31.729402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-b8698924-3e62-4ea4-a6bc-6cdbe58fcaa6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:09:31.729402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:31.729402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:31.729402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:31.729402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:31.729402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:31.729402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:31.729402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:31.729402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:31.729402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:31.729402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:31.729402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:31.729402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:31.729402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:31.729402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:31.729402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:31.730379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:31.730379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:31.730379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:31.730379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:31.730379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:31.730379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:31.730379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:31.730379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:31.730379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:31.730379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:31.730379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:31.730379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:31.730379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:31.730379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:31.730379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:31.730379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:31.731142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:31.731142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:31.731142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:31.731142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:31.731142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:31.731142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:31.731142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:31.731142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:31.731142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:31.731142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:31.731142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:09:31.731142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:09:31.731142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 16/30] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:09:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:31.732797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8993cd1-edbd-4a55-8a29-e7d8b2d5dbe8 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:31.732797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8993cd1-edbd-4a55-8a29-e7d8b2d5dbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:31.732797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8993cd1-edbd-4a55-8a29-e7d8b2d5dbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:31.759612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-c8993cd1-edbd-4a55-8a29-e7d8b2d5dbe8 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:31.761181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c8993cd1-edbd-4a55-8a29-e7d8b2d5dbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:31.801562 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:31.861095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 17/31] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:31.865804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27faf1e7-f941-4181-b992-b87bf92f2670 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:31.867605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8993cd1-edbd-4a55-8a29-e7d8b2d5dbe8 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:31.918465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f88412-81b2-4170-8d02-0c1cc35b42f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:31.919156 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f88412-81b2-4170-8d02-0c1cc35b42f6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:31.919541 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f88412-81b2-4170-8d02-0c1cc35b42f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:31.919842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f88412-81b2-4170-8d02-0c1cc35b42f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:31.923281 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:31.937329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f88412-81b2-4170-8d02-0c1cc35b42f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:31.939476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f88412-81b2-4170-8d02-0c1cc35b42f6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:31.964289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e0f88412-81b2-4170-8d02-0c1cc35b42f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:09:31.964289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:31.964289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:31.964289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:31.964289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:31.964289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:31.964289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:31.964289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:31.964289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:31.964289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:31.964289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:31.964289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:31.964289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:09:31.964289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:09:31.964289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:31.965213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:31.966066 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:09:31.966066 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:09:31.966066 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:09:31.966066 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:09:31.966066 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 15/32] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:09:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:31.966834 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f88412-81b2-4170-8d02-0c1cc35b42f6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:32.002186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8993cd1-edbd-4a55-8a29-e7d8b2d5dbe8 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:32.060343 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe018df-9793-40f5-8053-d6d08e748a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:32.060984 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe018df-9793-40f5-8053-d6d08e748a9c None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:32.061362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe018df-9793-40f5-8053-d6d08e748a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:32.061645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe018df-9793-40f5-8053-d6d08e748a9c None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:32.062462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe018df-9793-40f5-8053-d6d08e748a9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:32.062768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe018df-9793-40f5-8053-d6d08e748a9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:32.064239 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe018df-9793-40f5-8053-d6d08e748a9c None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:32.080099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 18/33] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:09:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:32.081630 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe018df-9793-40f5-8053-d6d08e748a9c None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:32.121451 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:32.128283 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f88412-81b2-4170-8d02-0c1cc35b42f6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:32.164989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae5278-220e-4f91-8a0a-5b28a6c6606b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:32.165611 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae5278-220e-4f91-8a0a-5b28a6c6606b None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:32.165945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae5278-220e-4f91-8a0a-5b28a6c6606b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:32.166226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae5278-220e-4f91-8a0a-5b28a6c6606b None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:32.166969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae5278-220e-4f91-8a0a-5b28a6c6606b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:32.168415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae5278-220e-4f91-8a0a-5b28a6c6606b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:32.189669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 16/34] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 1 15:09:31 2022] POST /identity/v3/roles => generated 225 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:09:32.190775 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae5278-220e-4f91-8a0a-5b28a6c6606b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:32.201186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe018df-9793-40f5-8053-d6d08e748a9c None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:32.241760 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d5fcd-8551-4b81-b515-fe390d1cb2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:32.242434 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d5fcd-8551-4b81-b515-fe390d1cb2ac None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:32.242767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d5fcd-8551-4b81-b515-fe390d1cb2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:32.242986 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d5fcd-8551-4b81-b515-fe390d1cb2ac None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:32.243717 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d5fcd-8551-4b81-b515-fe390d1cb2ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:32.244032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d5fcd-8551-4b81-b515-fe390d1cb2ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:32.245517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d5fcd-8551-4b81-b515-fe390d1cb2ac None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:32.253299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:32.259931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 19/35] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:32.265673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d5fcd-8551-4b81-b515-fe390d1cb2ac None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:32.329810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae5278-220e-4f91-8a0a-5b28a6c6606b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:32.380582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de1468-2083-4e77-91dd-43eb1c9c25e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:32.381302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de1468-2083-4e77-91dd-43eb1c9c25e0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:32.381590 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de1468-2083-4e77-91dd-43eb1c9c25e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:32.381901 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de1468-2083-4e77-91dd-43eb1c9c25e0 None admin] PATH_INFO: `/v3/users/1b4b6acdd88e421a86924fd2c155794f` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:32.383784 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de1468-2083-4e77-91dd-43eb1c9c25e0 None admin] RBAC: Authorizing `identity:get_user(user_id=1b4b6acdd88e421a86924fd2c155794f)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:32.385322 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de1468-2083-4e77-91dd-43eb1c9c25e0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:32.392045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 17/36] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:32 2022] GET /identity/v3/users/1b4b6acdd88e421a86924fd2c155794f => generated 247 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:32.401224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de1468-2083-4e77-91dd-43eb1c9c25e0 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:32.424285 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:32.442697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d5fcd-8551-4b81-b515-fe390d1cb2ac None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:32.485438 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffab092-9f26-417d-84fb-5875776a3c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:32.486415 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffab092-9f26-417d-84fb-5875776a3c98 None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:32.486892 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffab092-9f26-417d-84fb-5875776a3c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:32.487377 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffab092-9f26-417d-84fb-5875776a3c98 None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:32.488539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffab092-9f26-417d-84fb-5875776a3c98 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:32.490273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffab092-9f26-417d-84fb-5875776a3c98 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:32.555321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de1468-2083-4e77-91dd-43eb1c9c25e0 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:32.570605 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:32.596956 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97908e65-62c3-4edb-aa30-1959c83ffdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:32.597644 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97908e65-62c3-4edb-aa30-1959c83ffdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:32.597986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97908e65-62c3-4edb-aa30-1959c83ffdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:32.598320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97908e65-62c3-4edb-aa30-1959c83ffdd3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:32.599583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97908e65-62c3-4edb-aa30-1959c83ffdd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:32.601103 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97908e65-62c3-4edb-aa30-1959c83ffdd3 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:32.602887 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 18/37] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:09:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:32.609196 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97908e65-62c3-4edb-aa30-1959c83ffdd3 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:32.725880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97908e65-62c3-4edb-aa30-1959c83ffdd3 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:32.763033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945594d-8d06-4ace-807d-240ba3970c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:32.763752 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945594d-8d06-4ace-807d-240ba3970c94 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:32.764039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945594d-8d06-4ace-807d-240ba3970c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:32.764328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945594d-8d06-4ace-807d-240ba3970c94 None admin] PATH_INFO: `/v3/domains/default/users/1b4b6acdd88e421a86924fd2c155794f/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:32.765538 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 20/38] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 15:09:32 2022] POST /identity/v3/projects => generated 301 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:09:32.766533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945594d-8d06-4ace-807d-240ba3970c94 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1b4b6acdd88e421a86924fd2c155794f, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:32.767204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f47b0-2688-473d-bca3-4009f4672f2d None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:32.767502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f47b0-2688-473d-bca3-4009f4672f2d None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:32.767784 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f47b0-2688-473d-bca3-4009f4672f2d None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:32.768097 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945594d-8d06-4ace-807d-240ba3970c94 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:32.773536 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 21/39] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:32.773536 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934573a2-977d-4325-aa15-28e82c42507e None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:32.773536 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934573a2-977d-4325-aa15-28e82c42507e None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:32.773536 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934573a2-977d-4325-aa15-28e82c42507e None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:32.773536 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 22/40] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:32.779099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbfaa10-5003-42f2-bc40-f4934cbbb00a None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:32.779311 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbfaa10-5003-42f2-bc40-f4934cbbb00a None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:32.779595 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbfaa10-5003-42f2-bc40-f4934cbbb00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:32.790662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 19/41] 10.222.0.48 () {58 vars in 1361 bytes} [Mon Aug 1 15:09:32 2022] PUT /identity/v3/domains/default/users/1b4b6acdd88e421a86924fd2c155794f/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:09:32.791725 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9da4e-105c-45ba-80ff-68aad562a641 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:32.792017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9da4e-105c-45ba-80ff-68aad562a641 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:32.792293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9da4e-105c-45ba-80ff-68aad562a641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:32.800356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-6dbfaa10-5003-42f2-bc40-f4934cbbb00a None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:32.802146 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6dbfaa10-5003-42f2-bc40-f4934cbbb00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:32.812834 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-14c9da4e-105c-45ba-80ff-68aad562a641 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:32.814496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-14c9da4e-105c-45ba-80ff-68aad562a641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:32.857680 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:32.894863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 23/42] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:32.918563 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbfaa10-5003-42f2-bc40-f4934cbbb00a None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:32.927534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 20/43] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:32.951383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9da4e-105c-45ba-80ff-68aad562a641 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:33.023600 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:33.072952 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbfaa10-5003-42f2-bc40-f4934cbbb00a None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:33.095392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9da4e-105c-45ba-80ff-68aad562a641 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:33.115323 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5344698e-cf2f-4678-8b97-bdd52caae02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:33.116201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344698e-cf2f-4678-8b97-bdd52caae02a None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:33.117133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344698e-cf2f-4678-8b97-bdd52caae02a None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:33.117582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5344698e-cf2f-4678-8b97-bdd52caae02a None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:33.118752 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344698e-cf2f-4678-8b97-bdd52caae02a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:33.121359 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5344698e-cf2f-4678-8b97-bdd52caae02a None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:33.132271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c22af8-7960-427a-baee-04dcd540085f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:33.132974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c22af8-7960-427a-baee-04dcd540085f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:33.133258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c22af8-7960-427a-baee-04dcd540085f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:33.133627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c22af8-7960-427a-baee-04dcd540085f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:33.134923 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c22af8-7960-427a-baee-04dcd540085f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:33.136486 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c22af8-7960-427a-baee-04dcd540085f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:33.138462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 21/44] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:09:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:09:33.143273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 24/45] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 1 15:09:32 2022] POST /identity/v3/roles => generated 223 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:09:33.144172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c22af8-7960-427a-baee-04dcd540085f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:33.196638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:33.268025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c22af8-7960-427a-baee-04dcd540085f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:33.308691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ec453-3582-4caa-8488-3e6dc28f7c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:33.309165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ec453-3582-4caa-8488-3e6dc28f7c57 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:33.310087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ec453-3582-4caa-8488-3e6dc28f7c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:33.310087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ec453-3582-4caa-8488-3e6dc28f7c57 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:33.310547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ec453-3582-4caa-8488-3e6dc28f7c57 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:33.312095 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ec453-3582-4caa-8488-3e6dc28f7c57 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:33.344349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 22/46] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 15:09:33 2022] POST /identity/v3/projects => generated 312 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:09:34.890401 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c674ca-e2f9-4344-9017-8d5137686146 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:34.890912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c674ca-e2f9-4344-9017-8d5137686146 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:34.891035 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c674ca-e2f9-4344-9017-8d5137686146 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:34.892452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 25/47] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:34.897679 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32bc41-0faa-448c-9b43-3a2327462cb3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:34.898109 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32bc41-0faa-448c-9b43-3a2327462cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:34.898370 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32bc41-0faa-448c-9b43-3a2327462cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:34.919465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4e32bc41-0faa-448c-9b43-3a2327462cb3 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:34.920758 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4e32bc41-0faa-448c-9b43-3a2327462cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:35.001166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 23/48] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:35.035615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c674ca-e2f9-4344-9017-8d5137686146 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:35.183061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c674ca-e2f9-4344-9017-8d5137686146 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:35.221477 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:35.230408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0282c173-4d33-40d0-8894-c7ba474f45cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:35.231172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282c173-4d33-40d0-8894-c7ba474f45cd None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:35.231691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282c173-4d33-40d0-8894-c7ba474f45cd None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:35.231972 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0282c173-4d33-40d0-8894-c7ba474f45cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:35.245628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0282c173-4d33-40d0-8894-c7ba474f45cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:35.250853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-0282c173-4d33-40d0-8894-c7ba474f45cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:09:35.250853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:35.250853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:35.250853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:35.250853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:35.250853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:35.250853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:35.250853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:35.250853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:35.250853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:35.250853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:35.250853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:35.250853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:35.250853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:35.250853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:35.250853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:35.261264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:35.261264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:35.261264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:35.261264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:35.261264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:35.261264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:35.261264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:35.261264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:35.261264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:35.261264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:35.261264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:35.261264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:35.261264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:35.261264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:35.261264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:35.261264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:35.262854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:35.262854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:35.262854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:35.262854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:35.262854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:35.262854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:35.262854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:35.262854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:35.262854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:35.262854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:35.262854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:09:35.262854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:09:35.262854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 26/49] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:09:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:35.264100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32bc41-0faa-448c-9b43-3a2327462cb3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:35.305715 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:35.417337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516cc73-2f2d-4e95-ba4c-53f8b6f6b882 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:35.417611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516cc73-2f2d-4e95-ba4c-53f8b6f6b882 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:35.417731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32bc41-0faa-448c-9b43-3a2327462cb3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:35.417991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516cc73-2f2d-4e95-ba4c-53f8b6f6b882 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:35.419387 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 27/50] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:09:35.424812 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c0d3e-669b-4f0e-8cdf-01e3e97b9175 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:35.425100 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c0d3e-669b-4f0e-8cdf-01e3e97b9175 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:35.425386 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c0d3e-669b-4f0e-8cdf-01e3e97b9175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:35.447807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-db8c0d3e-669b-4f0e-8cdf-01e3e97b9175 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:35.449073 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-db8c0d3e-669b-4f0e-8cdf-01e3e97b9175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:35.459094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7315eac-a205-45cd-a59c-d15f8dd3d9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:35.459732 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7315eac-a205-45cd-a59c-d15f8dd3d9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:35.460015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7315eac-a205-45cd-a59c-d15f8dd3d9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:35.460288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7315eac-a205-45cd-a59c-d15f8dd3d9f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:35.461018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7315eac-a205-45cd-a59c-d15f8dd3d9f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:35.461323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7315eac-a205-45cd-a59c-d15f8dd3d9f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:35.462777 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7315eac-a205-45cd-a59c-d15f8dd3d9f0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:35.477420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 24/51] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:09:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:35.478865 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7315eac-a205-45cd-a59c-d15f8dd3d9f0 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:35.527401 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 28/52] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:09:35.550499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c0d3e-669b-4f0e-8cdf-01e3e97b9175 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:35.581594 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:35.599136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7315eac-a205-45cd-a59c-d15f8dd3d9f0 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:35.633653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3779fc3-4c60-4c2e-8809-6e6dfb8e16ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:35.634337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3779fc3-4c60-4c2e-8809-6e6dfb8e16ca None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:35.634616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3779fc3-4c60-4c2e-8809-6e6dfb8e16ca None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:35.634887 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3779fc3-4c60-4c2e-8809-6e6dfb8e16ca None admin] PATH_INFO: `/v3/users/1b4b6acdd88e421a86924fd2c155794f` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:35.636086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3779fc3-4c60-4c2e-8809-6e6dfb8e16ca None admin] RBAC: Authorizing `identity:get_user(user_id=1b4b6acdd88e421a86924fd2c155794f)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:35.637557 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3779fc3-4c60-4c2e-8809-6e6dfb8e16ca None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:35.639216 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 25/53] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:35 2022] GET /identity/v3/users/1b4b6acdd88e421a86924fd2c155794f => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:09:35.645007 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3779fc3-4c60-4c2e-8809-6e6dfb8e16ca None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:35.672901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c0d3e-669b-4f0e-8cdf-01e3e97b9175 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:35.694614 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:35.710980 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a37fa-ec02-4a4c-b2c7-e048b020b604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:35.711577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a37fa-ec02-4a4c-b2c7-e048b020b604 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:35.711858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a37fa-ec02-4a4c-b2c7-e048b020b604 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:35.712138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a37fa-ec02-4a4c-b2c7-e048b020b604 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:35.713327 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a37fa-ec02-4a4c-b2c7-e048b020b604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:35.714839 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a37fa-ec02-4a4c-b2c7-e048b020b604 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:35.716465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 29/54] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:09:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:09:35.722299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a37fa-ec02-4a4c-b2c7-e048b020b604 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:35.754779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3779fc3-4c60-4c2e-8809-6e6dfb8e16ca None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:35.795086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ab836-1099-4985-8c1c-e159f1ad8d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:35.795770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ab836-1099-4985-8c1c-e159f1ad8d45 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:35.796076 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ab836-1099-4985-8c1c-e159f1ad8d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:35.796351 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ab836-1099-4985-8c1c-e159f1ad8d45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:35.797554 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ab836-1099-4985-8c1c-e159f1ad8d45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:35.799040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ab836-1099-4985-8c1c-e159f1ad8d45 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:35.800648 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 26/55] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:09:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:09:35.806589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ab836-1099-4985-8c1c-e159f1ad8d45 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:35.833426 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a37fa-ec02-4a4c-b2c7-e048b020b604 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:35.837033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:35.872742 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d306d73-1cff-4c04-9800-c0a0d4e36ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:35.873393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d306d73-1cff-4c04-9800-c0a0d4e36ecf None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:35.873675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d306d73-1cff-4c04-9800-c0a0d4e36ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:35.873985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d306d73-1cff-4c04-9800-c0a0d4e36ecf None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:35.874699 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d306d73-1cff-4c04-9800-c0a0d4e36ecf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:35.876295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d306d73-1cff-4c04-9800-c0a0d4e36ecf None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:35.903505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 30/56] 10.222.0.48 () {60 vars in 1140 bytes} [Mon Aug 1 15:09:35 2022] POST /identity/v3/projects => generated 298 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:09:35.929691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ab836-1099-4985-8c1c-e159f1ad8d45 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:35.973648 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ddf21-7a93-4fd0-8bc0-272abc15c1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:35.974352 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ddf21-7a93-4fd0-8bc0-272abc15c1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:35.974657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ddf21-7a93-4fd0-8bc0-272abc15c1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:35.974905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ddf21-7a93-4fd0-8bc0-272abc15c1f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:35.975531 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ddf21-7a93-4fd0-8bc0-272abc15c1f5 None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.domain.id=default, user.id=1b4b6acdd88e421a86924fd2c155794f) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:35.975844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ddf21-7a93-4fd0-8bc0-272abc15c1f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:35.977367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ddf21-7a93-4fd0-8bc0-272abc15c1f5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:35.984921 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 27/57] 10.222.0.48 () {56 vars in 1322 bytes} [Mon Aug 1 15:09:35 2022] GET /identity/v3/role_assignments?user.id=1b4b6acdd88e421a86924fd2c155794f&scope.domain.id=default&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 530 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:09:37.935157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61849de7-cca1-4033-8547-5fcc47ad7193 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:37.935952 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61849de7-cca1-4033-8547-5fcc47ad7193 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:37.936102 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61849de7-cca1-4033-8547-5fcc47ad7193 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:37.938406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 31/58] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:37.945357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698542c-c383-480f-bf98-0e3653a1fefd None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:37.945826 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698542c-c383-480f-bf98-0e3653a1fefd None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:37.945915 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5698542c-c383-480f-bf98-0e3653a1fefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:37.967502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5698542c-c383-480f-bf98-0e3653a1fefd None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:37.968720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5698542c-c383-480f-bf98-0e3653a1fefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:38.044224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 28/59] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:09:38.068567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61849de7-cca1-4033-8547-5fcc47ad7193 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:38.117929 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:38.182579 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61849de7-cca1-4033-8547-5fcc47ad7193 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:38.219512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef979124-21bf-4282-8054-719e070fe661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:38.220167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef979124-21bf-4282-8054-719e070fe661 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:38.220447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef979124-21bf-4282-8054-719e070fe661 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:38.220692 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef979124-21bf-4282-8054-719e070fe661 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:38.222039 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef979124-21bf-4282-8054-719e070fe661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:38.224743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef979124-21bf-4282-8054-719e070fe661 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:38.226417 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 32/60] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:09:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:38.232528 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698542c-c383-480f-bf98-0e3653a1fefd None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:38.249476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:38.341563 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5698542c-c383-480f-bf98-0e3653a1fefd None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:38.382945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec00d1f-261d-40fb-afbe-b83c47afd637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:38.383567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec00d1f-261d-40fb-afbe-b83c47afd637 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:38.383842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec00d1f-261d-40fb-afbe-b83c47afd637 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:38.384114 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec00d1f-261d-40fb-afbe-b83c47afd637 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:38.384801 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec00d1f-261d-40fb-afbe-b83c47afd637 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:38.386301 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec00d1f-261d-40fb-afbe-b83c47afd637 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:38.414831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 29/61] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:09:38 2022] POST /identity/v3/users => generated 275 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:09:41.798723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b9b90-633f-496e-a942-be0577524a40 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:41.799287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b9b90-633f-496e-a942-be0577524a40 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:41.799287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b9b90-633f-496e-a942-be0577524a40 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:41.800618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 33/62] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:41.806100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721f732-c7c0-4478-9594-2eb32d58e95f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:41.806546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721f732-c7c0-4478-9594-2eb32d58e95f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:41.806649 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721f732-c7c0-4478-9594-2eb32d58e95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:41.831201 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2721f732-c7c0-4478-9594-2eb32d58e95f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:41.832537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2721f732-c7c0-4478-9594-2eb32d58e95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:41.926595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:41.935072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 30/63] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:41.962995 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b9b90-633f-496e-a942-be0577524a40 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:42.011602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8bf9e-95e7-463e-83af-b2836e03a567 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:42.011926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8bf9e-95e7-463e-83af-b2836e03a567 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:42.012153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8bf9e-95e7-463e-83af-b2836e03a567 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:42.014042 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 31/64] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:09:42.034884 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee226f2-a740-4591-9a7c-77578740984b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:42.035583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee226f2-a740-4591-9a7c-77578740984b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:42.036058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee226f2-a740-4591-9a7c-77578740984b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:42.079596 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6ee226f2-a740-4591-9a7c-77578740984b None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:42.080768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6ee226f2-a740-4591-9a7c-77578740984b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:42.108101 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b9b90-633f-496e-a942-be0577524a40 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:42.118348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:42.144952 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6f653-3357-478b-80fc-674516cbebb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:42.145953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6f653-3357-478b-80fc-674516cbebb9 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:42.146582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6f653-3357-478b-80fc-674516cbebb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:42.147072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6f653-3357-478b-80fc-674516cbebb9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:42.159243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6f653-3357-478b-80fc-674516cbebb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:42.160474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 32/65] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:09:42.161903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900007d-aeec-40d8-8a51-b4c77cab60e8 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:42.162256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900007d-aeec-40d8-8a51-b4c77cab60e8 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:42.162600 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900007d-aeec-40d8-8a51-b4c77cab60e8 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:42.163962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 33/66] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:42.166448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-5cb6f653-3357-478b-80fc-674516cbebb9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:09:42.166448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:42.166448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:42.166448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:42.166448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:42.166448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:42.166448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:42.166448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:42.166448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:42.166448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:42.166448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:42.166448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:42.166448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:42.166448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:42.166448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:42.166448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:42.167291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:42.167291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:42.167291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:42.167291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:42.167291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:42.167291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:42.167291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:42.167291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:42.167291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:42.167291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:42.167291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:42.167291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:42.167291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:42.167291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:42.167291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:42.167291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:42.168069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:42.168069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:42.168069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:42.168069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:42.168069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:42.168069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:42.168069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:42.168069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:42.168069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:42.168069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:42.168069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:09:42.168069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:09:42.169460 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 34/67] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:09:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:42.173817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bea2f-bd68-4c95-b367-945ff5ef1b33 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:42.173817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bea2f-bd68-4c95-b367-945ff5ef1b33 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:42.173817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53bea2f-bd68-4c95-b367-945ff5ef1b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:42.175947 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6f653-3357-478b-80fc-674516cbebb9 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:42.195371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d53bea2f-bd68-4c95-b367-945ff5ef1b33 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:42.196832 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d53bea2f-bd68-4c95-b367-945ff5ef1b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:42.287168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 34/68] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:42.288441 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bea2f-bd68-4c95-b367-945ff5ef1b33 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:42.297333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6f653-3357-478b-80fc-674516cbebb9 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:42.334649 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12531ce-d8e2-49aa-bdee-266e3416f7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:42.335216 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12531ce-d8e2-49aa-bdee-266e3416f7ef None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:42.335493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12531ce-d8e2-49aa-bdee-266e3416f7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:42.335774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12531ce-d8e2-49aa-bdee-266e3416f7ef None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:42.336520 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12531ce-d8e2-49aa-bdee-266e3416f7ef None admin] RBAC: Adding query filter params (name=member) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:42.336824 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12531ce-d8e2-49aa-bdee-266e3416f7ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:42.338281 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12531ce-d8e2-49aa-bdee-266e3416f7ef None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:42.347450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 35/69] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:09:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:09:42.349081 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370c0f5-4e3a-4123-9379-d1111d5a5ee5 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:42.349370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370c0f5-4e3a-4123-9379-d1111d5a5ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:42.349644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370c0f5-4e3a-4123-9379-d1111d5a5ee5 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:42.350946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 36/70] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:42.354248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370c0f5-4e3a-4123-9379-d1111d5a5ee5 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:42.358375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:42.393911 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:42.405933 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53bea2f-bd68-4c95-b367-945ff5ef1b33 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:42.443155 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52a8af-7fe1-4c4b-89d8-9a8556cda59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:42.443818 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52a8af-7fe1-4c4b-89d8-9a8556cda59c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:42.444103 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52a8af-7fe1-4c4b-89d8-9a8556cda59c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:42.444380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52a8af-7fe1-4c4b-89d8-9a8556cda59c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:42.455003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52a8af-7fe1-4c4b-89d8-9a8556cda59c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:42.460209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-8d52a8af-7fe1-4c4b-89d8-9a8556cda59c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:09:42.460209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:42.460209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:42.460209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:42.460209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:42.460209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:42.460209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:42.460209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:42.460209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:42.460209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:42.460209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:42.460209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:42.460209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:42.460209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:42.460209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:42.460209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:42.461131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:42.461131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:42.461131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:42.461131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:42.461131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:42.461131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:42.461131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:42.461131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:42.461131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:42.461131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:42.461131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:42.461131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:42.461131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:42.461131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:42.461131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:42.461131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:42.462214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:42.462214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:42.462214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:42.462214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:42.462214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:42.462214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:42.462214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:42.462214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:42.462214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:42.462214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:42.462214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:09:42.462214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:09:42.462214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 35/71] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:09:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:42.462214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52a8af-7fe1-4c4b-89d8-9a8556cda59c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:42.475121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370c0f5-4e3a-4123-9379-d1111d5a5ee5 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:42.511659 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6403d5-a4dc-41fa-8710-6347bc757ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:42.512325 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6403d5-a4dc-41fa-8710-6347bc757ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:42.512605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6403d5-a4dc-41fa-8710-6347bc757ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:42.512876 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6403d5-a4dc-41fa-8710-6347bc757ae4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:42.524872 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6403d5-a4dc-41fa-8710-6347bc757ae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:42.529923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-1d6403d5-a4dc-41fa-8710-6347bc757ae4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:09:42.529923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:42.529923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:42.529923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:42.529923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:42.529923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:42.529923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:42.529923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:42.529923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:42.529923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:42.529923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:42.529923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:42.529923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:42.529923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:42.529923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:42.529923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:42.530795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:42.530795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:42.530795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:42.530795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:42.530795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:42.530795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:42.530795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:42.530795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:42.530795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:42.530795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:42.530795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:42.530795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:42.530795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:42.530795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:42.530795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:42.530795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:42.531686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:42.531686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:42.531686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:42.531686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:42.531686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:42.531686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:42.531686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:42.531686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:42.531686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:42.531686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:42.531686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:09:42.531686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:09:42.531686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 37/72] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:09:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:42.532511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf5e9f-bb37-4c2d-a457-daae40fb3f16 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:42.532575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf5e9f-bb37-4c2d-a457-daae40fb3f16 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:42.532883 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf5e9f-bb37-4c2d-a457-daae40fb3f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:42.551794 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2bcf5e9f-bb37-4c2d-a457-daae40fb3f16 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:42.553024 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2bcf5e9f-bb37-4c2d-a457-daae40fb3f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:42.573132 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52a8af-7fe1-4c4b-89d8-9a8556cda59c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:42.595219 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:42.612409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99950dd5-78fc-4d2b-8785-839e51219afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:42.613051 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99950dd5-78fc-4d2b-8785-839e51219afc None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:42.613437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99950dd5-78fc-4d2b-8785-839e51219afc None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:42.613715 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99950dd5-78fc-4d2b-8785-839e51219afc None admin] PATH_INFO: `/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:42.627241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 38/73] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:42.627241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3dbf9-f7fb-4a29-a62a-27581cd40a18 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:42.627241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3dbf9-f7fb-4a29-a62a-27581cd40a18 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:42.627241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3dbf9-f7fb-4a29-a62a-27581cd40a18 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:42.630403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 39/74] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:42.631062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3dbf9-f7fb-4a29-a62a-27581cd40a18 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:42.632943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99950dd5-78fc-4d2b-8785-839e51219afc None admin] RBAC: Authorizing `identity:get_user(user_id=a0dfd1dc38b741ebadff99a9c1fa4e99)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:42.635533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99950dd5-78fc-4d2b-8785-839e51219afc None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:42.640648 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 36/75] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:42 2022] GET /identity/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:42.640648 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99950dd5-78fc-4d2b-8785-839e51219afc None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:42.667864 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:42.759977 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99950dd5-78fc-4d2b-8785-839e51219afc None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:42.771179 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3dbf9-f7fb-4a29-a62a-27581cd40a18 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:42.799757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301544d-7ac7-43f8-88c4-793cd63c3d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:42.800400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301544d-7ac7-43f8-88c4-793cd63c3d9a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:42.800682 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301544d-7ac7-43f8-88c4-793cd63c3d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:42.801038 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301544d-7ac7-43f8-88c4-793cd63c3d9a None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:42.801805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301544d-7ac7-43f8-88c4-793cd63c3d9a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:42.802128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301544d-7ac7-43f8-88c4-793cd63c3d9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:42.803555 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301544d-7ac7-43f8-88c4-793cd63c3d9a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:42.808808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1b510-4373-400c-a4c6-4729d564cfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:42.809615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1b510-4373-400c-a4c6-4729d564cfbc None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:42.809926 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1b510-4373-400c-a4c6-4729d564cfbc None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:42.810202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1b510-4373-400c-a4c6-4729d564cfbc None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:42.810936 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1b510-4373-400c-a4c6-4729d564cfbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:42.811382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1b510-4373-400c-a4c6-4729d564cfbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:42.812838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1b510-4373-400c-a4c6-4729d564cfbc None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:42.816662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 37/76] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:09:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:42.823562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9ecfb-f7b6-4eb5-a39e-b117f1d031a1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:42.823562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9ecfb-f7b6-4eb5-a39e-b117f1d031a1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:42.823562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9ecfb-f7b6-4eb5-a39e-b117f1d031a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:42.828143 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 40/77] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:09:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:42.829108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1b510-4373-400c-a4c6-4729d564cfbc None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:42.842992 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-85a9ecfb-f7b6-4eb5-a39e-b117f1d031a1 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:42.844242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-85a9ecfb-f7b6-4eb5-a39e-b117f1d031a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:42.883615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:42.923025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 38/78] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:42.923979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9ecfb-f7b6-4eb5-a39e-b117f1d031a1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:42.947715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:42.971493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1b510-4373-400c-a4c6-4729d564cfbc None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:43.011207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1397fa-c8d9-40bd-9eec-766482880566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:43.011207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1397fa-c8d9-40bd-9eec-766482880566 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:43.011650 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1397fa-c8d9-40bd-9eec-766482880566 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:43.011754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1397fa-c8d9-40bd-9eec-766482880566 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:43.049633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9ecfb-f7b6-4eb5-a39e-b117f1d031a1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:43.093226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9272a7a-8384-4cf3-b99e-b23540e25d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:43.093940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9272a7a-8384-4cf3-b99e-b23540e25d17 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:43.094226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9272a7a-8384-4cf3-b99e-b23540e25d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:43.094509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9272a7a-8384-4cf3-b99e-b23540e25d17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:43.105584 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9272a7a-8384-4cf3-b99e-b23540e25d17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:43.111331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d9272a7a-8384-4cf3-b99e-b23540e25d17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:09:43.111331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:43.111331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:43.111331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:43.111331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:43.111331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:43.111331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:43.111331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:43.111331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:43.111331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:43.111331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:43.111331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:43.111331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:43.111331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:43.111331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:43.111331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:43.112220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:43.112220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:43.112220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:43.112220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:43.112220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:43.112220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:43.112220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:43.112220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:43.112220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:43.112220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:43.112220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:43.112220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:43.112220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:43.112220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:43.112220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:43.112220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:43.113311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:43.113311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:43.113311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:43.113311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:43.113311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:43.113311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:43.113311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:43.113311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:43.113311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:43.113311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:43.113311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:09:43.113311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:09:43.113311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 39/79] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:09:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:43.114197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9272a7a-8384-4cf3-b99e-b23540e25d17 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:43.132637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1397fa-c8d9-40bd-9eec-766482880566 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:43.134716 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1397fa-c8d9-40bd-9eec-766482880566 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:43.150032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:43.228705 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9272a7a-8384-4cf3-b99e-b23540e25d17 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:43.228915 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:43.249522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 41/80] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:09:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:09:43.250429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1397fa-c8d9-40bd-9eec-766482880566 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:43.269816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda81ae9-4ce7-483b-97cd-7251039b56ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:43.270486 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81ae9-4ce7-483b-97cd-7251039b56ad None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:43.270763 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81ae9-4ce7-483b-97cd-7251039b56ad None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:43.271034 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda81ae9-4ce7-483b-97cd-7251039b56ad None admin] PATH_INFO: `/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:43.272224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda81ae9-4ce7-483b-97cd-7251039b56ad None admin] RBAC: Authorizing `identity:get_user(user_id=a0dfd1dc38b741ebadff99a9c1fa4e99)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:43.273765 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda81ae9-4ce7-483b-97cd-7251039b56ad None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:43.275381 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 40/81] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:43 2022] GET /identity/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:43.276150 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda81ae9-4ce7-483b-97cd-7251039b56ad None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:43.363708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1397fa-c8d9-40bd-9eec-766482880566 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:43.392274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:43.396322 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda81ae9-4ce7-483b-97cd-7251039b56ad None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:43.406352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b8d24-f894-431c-861b-41396987e38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:43.407034 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b8d24-f894-431c-861b-41396987e38d None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:43.407317 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b8d24-f894-431c-861b-41396987e38d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:43.407597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b8d24-f894-431c-861b-41396987e38d None admin] PATH_INFO: `/v3/users/1b4b6acdd88e421a86924fd2c155794f` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:43.409061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b8d24-f894-431c-861b-41396987e38d None admin] RBAC: Authorizing `identity:get_user(user_id=1b4b6acdd88e421a86924fd2c155794f)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:43.410628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b8d24-f894-431c-861b-41396987e38d None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:43.412259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 42/82] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:43 2022] GET /identity/v3/users/1b4b6acdd88e421a86924fd2c155794f => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:43.413027 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b8d24-f894-431c-861b-41396987e38d None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:43.438928 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45d574-ba25-482c-91fd-1919e25a827c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:43.439595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45d574-ba25-482c-91fd-1919e25a827c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:43.439880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45d574-ba25-482c-91fd-1919e25a827c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:43.440152 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45d574-ba25-482c-91fd-1919e25a827c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:43.441799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45d574-ba25-482c-91fd-1919e25a827c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:43.443270 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45d574-ba25-482c-91fd-1919e25a827c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:43.444895 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 41/83] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:09:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:43.450247 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45d574-ba25-482c-91fd-1919e25a827c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:43.473037 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:43.526960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b8d24-f894-431c-861b-41396987e38d None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:43.550150 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45d574-ba25-482c-91fd-1919e25a827c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:43.562162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af292757-c7a5-4424-b24f-728e9e6c4b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:43.562790 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af292757-c7a5-4424-b24f-728e9e6c4b76 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:43.563078 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af292757-c7a5-4424-b24f-728e9e6c4b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:43.563361 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af292757-c7a5-4424-b24f-728e9e6c4b76 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:43.564097 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af292757-c7a5-4424-b24f-728e9e6c4b76 None admin] RBAC: Adding query filter params (name=member) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:43.564394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af292757-c7a5-4424-b24f-728e9e6c4b76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:43.565872 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af292757-c7a5-4424-b24f-728e9e6c4b76 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:43.574206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 43/84] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:09:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:43.575235 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af292757-c7a5-4424-b24f-728e9e6c4b76 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:43.585637 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b9b40-0afa-4a2f-b908-aeb090aaf3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:43.586326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b9b40-0afa-4a2f-b908-aeb090aaf3ad None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:43.586620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b9b40-0afa-4a2f-b908-aeb090aaf3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:43.586878 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b9b40-0afa-4a2f-b908-aeb090aaf3ad None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:43.588040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b9b40-0afa-4a2f-b908-aeb090aaf3ad None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:43.589502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b9b40-0afa-4a2f-b908-aeb090aaf3ad None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:43.591201 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 42/85] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:09:43 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:43.596656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b9b40-0afa-4a2f-b908-aeb090aaf3ad None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:43.627585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:43.679825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af292757-c7a5-4424-b24f-728e9e6c4b76 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:43.692632 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:43.695347 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b9b40-0afa-4a2f-b908-aeb090aaf3ad None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:43.715314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3f628-6d9d-4cd0-8cc8-9976c23a8e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:43.715946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3f628-6d9d-4cd0-8cc8-9976c23a8e96 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:43.716225 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3f628-6d9d-4cd0-8cc8-9976c23a8e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:43.716498 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3f628-6d9d-4cd0-8cc8-9976c23a8e96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:43.728825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55abe8ae-0ad8-41c6-9711-67f5c22d97b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:43.729430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abe8ae-0ad8-41c6-9711-67f5c22d97b3 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:43.729712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abe8ae-0ad8-41c6-9711-67f5c22d97b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:43.730013 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abe8ae-0ad8-41c6-9711-67f5c22d97b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:43.785807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3f628-6d9d-4cd0-8cc8-9976c23a8e96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:43.787357 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3f628-6d9d-4cd0-8cc8-9976c23a8e96 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:43.800623 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55abe8ae-0ad8-41c6-9711-67f5c22d97b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:43.802164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55abe8ae-0ad8-41c6-9711-67f5c22d97b3 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:43.834973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:43.892841 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 44/86] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:09:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:09:43.893682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3f628-6d9d-4cd0-8cc8-9976c23a8e96 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:43.907515 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:43.909767 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 43/87] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:09:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:09:43.910676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55abe8ae-0ad8-41c6-9711-67f5c22d97b3 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:43.996433 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3f628-6d9d-4cd0-8cc8-9976c23a8e96 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:44.010642 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55abe8ae-0ad8-41c6-9711-67f5c22d97b3 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:44.031757 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165b26a-3f94-48bd-8c70-7a86a709d905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:44.032398 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165b26a-3f94-48bd-8c70-7a86a709d905 None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:44.032682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165b26a-3f94-48bd-8c70-7a86a709d905 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:44.032962 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165b26a-3f94-48bd-8c70-7a86a709d905 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:44.033650 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165b26a-3f94-48bd-8c70-7a86a709d905 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:44.035165 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165b26a-3f94-48bd-8c70-7a86a709d905 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:44.043956 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705ef90-fb30-4f6f-b1d6-d32e150e667c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:44.044634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705ef90-fb30-4f6f-b1d6-d32e150e667c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:44.044920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705ef90-fb30-4f6f-b1d6-d32e150e667c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:44.045182 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705ef90-fb30-4f6f-b1d6-d32e150e667c None admin] PATH_INFO: `/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:44.046718 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705ef90-fb30-4f6f-b1d6-d32e150e667c None admin] RBAC: Authorizing `identity:get_user(user_id=a0dfd1dc38b741ebadff99a9c1fa4e99)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:44.047869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705ef90-fb30-4f6f-b1d6-d32e150e667c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:44.049493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 44/88] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:43 2022] GET /identity/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:44.050349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705ef90-fb30-4f6f-b1d6-d32e150e667c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:44.053480 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:44.066248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 45/89] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:09:43 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:09:44.067094 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165b26a-3f94-48bd-8c70-7a86a709d905 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:44.151079 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:44.196126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165b26a-3f94-48bd-8c70-7a86a709d905 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:44.206659 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705ef90-fb30-4f6f-b1d6-d32e150e667c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:44.235975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd685dc-c4b9-4285-b966-11fa57409715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:44.236606 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd685dc-c4b9-4285-b966-11fa57409715 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:44.236891 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd685dc-c4b9-4285-b966-11fa57409715 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:44.237171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd685dc-c4b9-4285-b966-11fa57409715 None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:44.239108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd685dc-c4b9-4285-b966-11fa57409715 None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:44.241213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd685dc-c4b9-4285-b966-11fa57409715 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:44.243435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 46/90] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:09:44 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:44.244516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd685dc-c4b9-4285-b966-11fa57409715 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:44.262797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ae6e2-d2b2-4190-b3e8-1da2cce10857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:44.263462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ae6e2-d2b2-4190-b3e8-1da2cce10857 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:44.263741 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ae6e2-d2b2-4190-b3e8-1da2cce10857 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:44.264025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ae6e2-d2b2-4190-b3e8-1da2cce10857 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:44.264852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ae6e2-d2b2-4190-b3e8-1da2cce10857 None admin] RBAC: Adding query filter params (role.id=19db9fb1cb1541ee94673df232bd6952, scope.project.id=ab228684d11649dfa97b4acebd727e17, user.id=a0dfd1dc38b741ebadff99a9c1fa4e99) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:44.265247 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ae6e2-d2b2-4190-b3e8-1da2cce10857 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:44.266787 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ae6e2-d2b2-4190-b3e8-1da2cce10857 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:44.274208 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 45/91] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:09:44 2022] GET /identity/v3/role_assignments?user.id=a0dfd1dc38b741ebadff99a9c1fa4e99&scope.project.id=ab228684d11649dfa97b4acebd727e17&role.id=19db9fb1cb1541ee94673df232bd6952 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:44.275158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ae6e2-d2b2-4190-b3e8-1da2cce10857 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:44.382117 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:44.405033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd685dc-c4b9-4285-b966-11fa57409715 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:44.447658 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ae6e2-d2b2-4190-b3e8-1da2cce10857 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:44.462381 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ecc70-e339-41c2-b328-5ff2a11887c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:44.463084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ecc70-e339-41c2-b328-5ff2a11887c9 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:44.463368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ecc70-e339-41c2-b328-5ff2a11887c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:44.463649 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ecc70-e339-41c2-b328-5ff2a11887c9 None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:44.465143 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ecc70-e339-41c2-b328-5ff2a11887c9 None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:44.466774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ecc70-e339-41c2-b328-5ff2a11887c9 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:44.468688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 47/92] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:09:44 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:44.469883 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ecc70-e339-41c2-b328-5ff2a11887c9 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:44.475167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:44.509943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9c74b-35f2-4b2c-ad34-c93b39865879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:44.510911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9c74b-35f2-4b2c-ad34-c93b39865879 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:44.511198 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9c74b-35f2-4b2c-ad34-c93b39865879 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:44.511480 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9c74b-35f2-4b2c-ad34-c93b39865879 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:44.597662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9c74b-35f2-4b2c-ad34-c93b39865879 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:44.599411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9c74b-35f2-4b2c-ad34-c93b39865879 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:44.605543 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ecc70-e339-41c2-b328-5ff2a11887c9 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:44.652034 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23965764-12ab-4124-a526-a5d5552155b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:44.652797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23965764-12ab-4124-a526-a5d5552155b6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:44.653079 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23965764-12ab-4124-a526-a5d5552155b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:44.653511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23965764-12ab-4124-a526-a5d5552155b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:44.654952 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23965764-12ab-4124-a526-a5d5552155b6 None admin] RBAC: Adding query filter params (role.id=7ae6dccec1244987a5bc7bf5c3991703, scope.project.id=ab228684d11649dfa97b4acebd727e17, user.id=a0dfd1dc38b741ebadff99a9c1fa4e99) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:44.655267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23965764-12ab-4124-a526-a5d5552155b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:44.657021 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23965764-12ab-4124-a526-a5d5552155b6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:44.680475 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 48/93] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:09:44 2022] GET /identity/v3/role_assignments?user.id=a0dfd1dc38b741ebadff99a9c1fa4e99&scope.project.id=ab228684d11649dfa97b4acebd727e17&role.id=7ae6dccec1244987a5bc7bf5c3991703 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:44.681435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23965764-12ab-4124-a526-a5d5552155b6 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:44.689427 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:44.796646 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 46/94] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:09:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:09:44.802447 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9c74b-35f2-4b2c-ad34-c93b39865879 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:44.823025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:44.886686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23965764-12ab-4124-a526-a5d5552155b6 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:44.959808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c06a36-5d59-4177-b55d-055205e8ed0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:44.960494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c06a36-5d59-4177-b55d-055205e8ed0b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:44.960788 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c06a36-5d59-4177-b55d-055205e8ed0b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:44.961010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c06a36-5d59-4177-b55d-055205e8ed0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:44.961921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c06a36-5d59-4177-b55d-055205e8ed0b None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=ab228684d11649dfa97b4acebd727e17, user.id=1b4b6acdd88e421a86924fd2c155794f) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:44.962244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c06a36-5d59-4177-b55d-055205e8ed0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:44.963878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c06a36-5d59-4177-b55d-055205e8ed0b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:44.978065 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 49/95] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:09:44 2022] GET /identity/v3/role_assignments?user.id=1b4b6acdd88e421a86924fd2c155794f&scope.project.id=ab228684d11649dfa97b4acebd727e17&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 259 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:45.033085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9c74b-35f2-4b2c-ad34-c93b39865879 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:45.090364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01632859-701b-41ea-8d93-9347804e3ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:45.091056 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01632859-701b-41ea-8d93-9347804e3ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:45.091317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01632859-701b-41ea-8d93-9347804e3ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:45.091589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01632859-701b-41ea-8d93-9347804e3ac9 None admin] PATH_INFO: `/v3/projects/541d4aab26d64344bf63c9afdab2dc6f` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:45.094769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01632859-701b-41ea-8d93-9347804e3ac9 None admin] RBAC: Authorizing `identity:get_project(project_id=541d4aab26d64344bf63c9afdab2dc6f)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:45.096894 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01632859-701b-41ea-8d93-9347804e3ac9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:45.100743 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 47/96] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:09:44 2022] GET /identity/v3/projects/541d4aab26d64344bf63c9afdab2dc6f => generated 312 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:09:45.106509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c06a36-5d59-4177-b55d-055205e8ed0b None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:45.278808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c06a36-5d59-4177-b55d-055205e8ed0b None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:45.322812 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:45.328732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625c37d-76a5-41de-b9b0-d4e9178e545d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:45.329386 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625c37d-76a5-41de-b9b0-d4e9178e545d None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:45.329663 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625c37d-76a5-41de-b9b0-d4e9178e545d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:45.329977 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625c37d-76a5-41de-b9b0-d4e9178e545d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:45.330808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625c37d-76a5-41de-b9b0-d4e9178e545d None admin] RBAC: Adding query filter params (role.id=19db9fb1cb1541ee94673df232bd6952, scope.project.id=541d4aab26d64344bf63c9afdab2dc6f, user.id=a0dfd1dc38b741ebadff99a9c1fa4e99) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:45.331122 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625c37d-76a5-41de-b9b0-d4e9178e545d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:45.332655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625c37d-76a5-41de-b9b0-d4e9178e545d None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:45.343671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 50/97] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:09:45 2022] GET /identity/v3/role_assignments?user.id=a0dfd1dc38b741ebadff99a9c1fa4e99&scope.project.id=541d4aab26d64344bf63c9afdab2dc6f&role.id=19db9fb1cb1541ee94673df232bd6952 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:46.795460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85906725-464b-4f38-8ac5-b57b8f076384 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:46.796503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85906725-464b-4f38-8ac5-b57b8f076384 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:46.797016 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85906725-464b-4f38-8ac5-b57b8f076384 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:46.798732 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 48/98] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:46.804384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f39fc-b935-4e01-83a3-1482dd95582a None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:46.805348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f39fc-b935-4e01-83a3-1482dd95582a None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:46.805848 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f39fc-b935-4e01-83a3-1482dd95582a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:46.828971 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-3c6f39fc-b935-4e01-83a3-1482dd95582a None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:46.830700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-3c6f39fc-b935-4e01-83a3-1482dd95582a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:46.938911 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 51/99] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:46.962503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85906725-464b-4f38-8ac5-b57b8f076384 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:47.010374 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:47.080236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85906725-464b-4f38-8ac5-b57b8f076384 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:47.123638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d7a37-2fa7-4e4e-aba6-9e2781558bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:47.124673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d7a37-2fa7-4e4e-aba6-9e2781558bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:47.125148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d7a37-2fa7-4e4e-aba6-9e2781558bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:47.125635 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d7a37-2fa7-4e4e-aba6-9e2781558bb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:47.130178 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d7a37-2fa7-4e4e-aba6-9e2781558bb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:47.132032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d7a37-2fa7-4e4e-aba6-9e2781558bb1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:47.134215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 49/100] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:09:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:47.140331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f39fc-b935-4e01-83a3-1482dd95582a None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:47.286023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f39fc-b935-4e01-83a3-1482dd95582a None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:47.319494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:47.328889 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6b755-db99-4dae-bf29-7c09b6d7c47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:47.329533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6b755-db99-4dae-bf29-7c09b6d7c47c None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:47.329851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6b755-db99-4dae-bf29-7c09b6d7c47c None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:47.330135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6b755-db99-4dae-bf29-7c09b6d7c47c None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:47.330899 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6b755-db99-4dae-bf29-7c09b6d7c47c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:47.332413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6b755-db99-4dae-bf29-7c09b6d7c47c None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:47.377739 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 52/101] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 15:09:47 2022] POST /identity/v3/projects => generated 302 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:09:47.535952 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d242b91-2780-4d18-b955-5a494fc909a3 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:47.536635 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d242b91-2780-4d18-b955-5a494fc909a3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:47.537114 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d242b91-2780-4d18-b955-5a494fc909a3 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:47.538713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 50/102] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:47.544213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc053d9a-edce-4b11-9c18-34ba83d06669 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:47.544854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc053d9a-edce-4b11-9c18-34ba83d06669 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:47.545223 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc053d9a-edce-4b11-9c18-34ba83d06669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:47.577914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-cc053d9a-edce-4b11-9c18-34ba83d06669 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:47.579200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-cc053d9a-edce-4b11-9c18-34ba83d06669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:47.583980 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edbc26-61e4-489c-acec-36de6c931580 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:47.584258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edbc26-61e4-489c-acec-36de6c931580 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:47.584450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edbc26-61e4-489c-acec-36de6c931580 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:47.585994 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 51/103] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:47.591376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d056e9-3fb5-4d50-a81c-e8205bc7b3f6 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:47.591670 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d056e9-3fb5-4d50-a81c-e8205bc7b3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:47.591931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d056e9-3fb5-4d50-a81c-e8205bc7b3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:47.634369 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-47d056e9-3fb5-4d50-a81c-e8205bc7b3f6 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:47.636403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-47d056e9-3fb5-4d50-a81c-e8205bc7b3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:47.687130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 53/104] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:09:47.695139 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55852833-8f46-48be-94c0-edcfdefca181 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:47.695771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55852833-8f46-48be-94c0-edcfdefca181 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:47.696252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55852833-8f46-48be-94c0-edcfdefca181 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:47.697936 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 54/105] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:47.713835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78a2e5-d031-435a-813d-7acd9e0a8aed None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:47.714472 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78a2e5-d031-435a-813d-7acd9e0a8aed None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:47.715010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78a2e5-d031-435a-813d-7acd9e0a8aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:47.739731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-dc78a2e5-d031-435a-813d-7acd9e0a8aed None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:47.741389 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-dc78a2e5-d031-435a-813d-7acd9e0a8aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:47.744275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 52/106] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:09:47.745086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d056e9-3fb5-4d50-a81c-e8205bc7b3f6 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:47.795225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:47.830910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 55/107] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:47.832835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78a2e5-d031-435a-813d-7acd9e0a8aed None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:47.896452 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d056e9-3fb5-4d50-a81c-e8205bc7b3f6 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:47.907443 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:47.974562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045656c-bfca-4a61-b5a7-c4b1d8e19677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:47.975359 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045656c-bfca-4a61-b5a7-c4b1d8e19677 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:47.975745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045656c-bfca-4a61-b5a7-c4b1d8e19677 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:47.976067 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045656c-bfca-4a61-b5a7-c4b1d8e19677 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:47.990958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78a2e5-d031-435a-813d-7acd9e0a8aed None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:48.007007 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0045656c-bfca-4a61-b5a7-c4b1d8e19677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:48.013400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-0045656c-bfca-4a61-b5a7-c4b1d8e19677 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:09:48.013400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:48.013400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:48.013400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:48.013400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:48.013400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:48.013400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:48.013400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:48.013400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:48.013400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:48.013400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:48.013400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:48.013400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:48.013400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:48.013400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:48.013400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:48.014593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:48.014593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:48.014593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:48.014593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:48.014593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:48.014593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:48.014593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:48.014593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:48.014593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:48.014593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:48.014593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:48.014593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:48.014593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:48.014593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:48.014593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:48.014593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:48.015382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:48.015382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:48.015382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:48.015382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:48.015382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:48.015382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:48.015382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:48.015382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:48.015382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:48.015382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:48.015382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:09:48.015382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:09:48.020403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 53/108] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:09:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:48.021349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045656c-bfca-4a61-b5a7-c4b1d8e19677 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:48.073506 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9eb5d-fe91-411e-aa2b-5da884f03cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:48.074718 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9eb5d-fe91-411e-aa2b-5da884f03cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:48.075218 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9eb5d-fe91-411e-aa2b-5da884f03cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:48.075707 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c9eb5d-fe91-411e-aa2b-5da884f03cd5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:48.117515 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c9eb5d-fe91-411e-aa2b-5da884f03cd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:48.135726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-d7c9eb5d-fe91-411e-aa2b-5da884f03cd5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:09:48.135726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:48.135726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:48.135726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:48.135726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:48.135726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:48.135726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:48.135726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:48.135726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:48.135726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:48.135726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:48.135726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:48.135726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:48.135726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:48.135726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:48.135726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:48.136651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:48.136651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:48.136651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:48.136651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:48.136651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:48.136651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:48.136651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:48.136651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:48.136651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:48.136651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:48.136651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:48.136651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:48.136651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:48.136651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:48.136651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:48.136651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:48.137512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:48.137512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:48.137512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:48.137512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:48.137512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:48.137512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:48.137512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:48.137512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:48.137512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:48.137512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:48.137512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:09:48.137512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:09:48.138135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 56/109] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:09:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:48.139582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9eb5d-fe91-411e-aa2b-5da884f03cd5 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:48.332541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c9eb5d-fe91-411e-aa2b-5da884f03cd5 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:48.338142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:48.340510 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:48.345472 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0045656c-bfca-4a61-b5a7-c4b1d8e19677 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:48.387015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d94c0-e4f4-4933-87b6-8af7594bc2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:48.387602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d94c0-e4f4-4933-87b6-8af7594bc2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:48.387999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d94c0-e4f4-4933-87b6-8af7594bc2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:48.388291 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d94c0-e4f4-4933-87b6-8af7594bc2a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:48.400125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d94c0-e4f4-4933-87b6-8af7594bc2a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:48.405391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-365d94c0-e4f4-4933-87b6-8af7594bc2a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:09:48.405391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:48.405391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:48.405391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:48.405391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:48.405391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:48.405391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:48.405391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:48.405391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:48.405391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:48.405391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:48.405391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:48.405391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:48.405391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:48.405391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:48.405391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:48.406333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:48.406333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:48.406333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:48.406333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:48.406333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:48.406333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:48.406333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:48.406333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:48.406333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:48.406333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:48.406333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:48.406333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:48.406333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:48.406333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:48.406333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:48.406333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:48.407099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:48.407099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:48.407099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:48.407099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:48.407099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:48.407099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:48.407099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:48.407099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:48.407099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:48.407099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:48.407099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:09:48.407099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:09:48.407099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 54/110] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:09:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:48.407744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d94c0-e4f4-4933-87b6-8af7594bc2a3 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:48.452176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f91ac6-730f-4a65-9303-caf2f28f5741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:48.453290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f91ac6-730f-4a65-9303-caf2f28f5741 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:48.453808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f91ac6-730f-4a65-9303-caf2f28f5741 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:48.454318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f91ac6-730f-4a65-9303-caf2f28f5741 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:48.455592 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f91ac6-730f-4a65-9303-caf2f28f5741 None admin] RBAC: Adding query filter params (name=member) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:48.456081 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f91ac6-730f-4a65-9303-caf2f28f5741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:48.458781 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f91ac6-730f-4a65-9303-caf2f28f5741 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:48.489329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 57/111] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:09:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:48.490824 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f91ac6-730f-4a65-9303-caf2f28f5741 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:48.523047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d94c0-e4f4-4933-87b6-8af7594bc2a3 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:48.562471 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf28ad-6e14-4485-8ffb-f78a1a7ac9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:48.563572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf28ad-6e14-4485-8ffb-f78a1a7ac9de None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:48.564403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf28ad-6e14-4485-8ffb-f78a1a7ac9de None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:48.564781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf28ad-6e14-4485-8ffb-f78a1a7ac9de None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:48.565943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf28ad-6e14-4485-8ffb-f78a1a7ac9de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:48.566388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf28ad-6e14-4485-8ffb-f78a1a7ac9de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:48.567920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf28ad-6e14-4485-8ffb-f78a1a7ac9de None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:48.581304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 55/112] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:09:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:48.582439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf28ad-6e14-4485-8ffb-f78a1a7ac9de None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:48.600994 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:48.643888 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f91ac6-730f-4a65-9303-caf2f28f5741 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:48.684651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117fe96-e106-43a6-8d93-a804bdbb6db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:48.685733 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117fe96-e106-43a6-8d93-a804bdbb6db0 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:48.686147 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117fe96-e106-43a6-8d93-a804bdbb6db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:48.686767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117fe96-e106-43a6-8d93-a804bdbb6db0 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:48.688293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117fe96-e106-43a6-8d93-a804bdbb6db0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:48.688710 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117fe96-e106-43a6-8d93-a804bdbb6db0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:48.690604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117fe96-e106-43a6-8d93-a804bdbb6db0 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:48.697756 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf28ad-6e14-4485-8ffb-f78a1a7ac9de None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:48.700813 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 58/113] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:09:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:48.702130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b8e37-297e-4a6a-bba5-001617b6f446 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:48.702427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b8e37-297e-4a6a-bba5-001617b6f446 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:48.702699 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b8e37-297e-4a6a-bba5-001617b6f446 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:48.704183 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 59/114] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:48.704913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b8e37-297e-4a6a-bba5-001617b6f446 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:48.733130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:48.743079 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da0f89-4d44-4bc0-9bba-9ed7a96c2caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:48.743717 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da0f89-4d44-4bc0-9bba-9ed7a96c2caf None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:48.744075 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da0f89-4d44-4bc0-9bba-9ed7a96c2caf None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:48.744329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da0f89-4d44-4bc0-9bba-9ed7a96c2caf None admin] PATH_INFO: `/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:48.745594 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da0f89-4d44-4bc0-9bba-9ed7a96c2caf None admin] RBAC: Authorizing `identity:get_user(user_id=a0dfd1dc38b741ebadff99a9c1fa4e99)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:48.746994 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da0f89-4d44-4bc0-9bba-9ed7a96c2caf None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:48.749380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 56/115] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:48 2022] GET /identity/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:48.749503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da0f89-4d44-4bc0-9bba-9ed7a96c2caf None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:48.824096 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b8e37-297e-4a6a-bba5-001617b6f446 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:48.848145 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:48.860156 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bc5c5-4344-4892-85bf-7647afe6d55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:48.860804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bc5c5-4344-4892-85bf-7647afe6d55b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:48.861022 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bc5c5-4344-4892-85bf-7647afe6d55b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:48.861299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bc5c5-4344-4892-85bf-7647afe6d55b None admin] PATH_INFO: `/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:48.862533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169bc5c5-4344-4892-85bf-7647afe6d55b None admin] RBAC: Authorizing `identity:get_user(user_id=a0dfd1dc38b741ebadff99a9c1fa4e99)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:48.864082 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169bc5c5-4344-4892-85bf-7647afe6d55b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:48.865756 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 60/116] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:48 2022] GET /identity/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:48.867191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506725e-6eae-4672-b0b5-f2021ec5fc6a None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:48.867459 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506725e-6eae-4672-b0b5-f2021ec5fc6a None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:48.867733 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506725e-6eae-4672-b0b5-f2021ec5fc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:48.876614 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da0f89-4d44-4bc0-9bba-9ed7a96c2caf None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:48.890298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2506725e-6eae-4672-b0b5-f2021ec5fc6a None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:48.891715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2506725e-6eae-4672-b0b5-f2021ec5fc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:48.914894 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200246d4-27e7-4119-860b-9c96d0d2606a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:48.915490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200246d4-27e7-4119-860b-9c96d0d2606a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:48.915695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200246d4-27e7-4119-860b-9c96d0d2606a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:48.916005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200246d4-27e7-4119-860b-9c96d0d2606a None admin] PATH_INFO: `/v3/users/f526a3c1fd564b2d9d3bec4abdac8135` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:48.931511 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200246d4-27e7-4119-860b-9c96d0d2606a None admin] RBAC: Authorizing `identity:get_user(user_id=f526a3c1fd564b2d9d3bec4abdac8135)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:48.933012 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200246d4-27e7-4119-860b-9c96d0d2606a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:48.939636 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 57/117] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:48 2022] GET /identity/v3/users/f526a3c1fd564b2d9d3bec4abdac8135 => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:48.939636 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847f483-8f98-43b1-ae6e-fc1a07ebb068 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:48.939636 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847f483-8f98-43b1-ae6e-fc1a07ebb068 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:48.939636 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847f483-8f98-43b1-ae6e-fc1a07ebb068 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:48.939636 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 58/118] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:48.939636 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847f483-8f98-43b1-ae6e-fc1a07ebb068 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:48.977803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 61/119] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:48.977803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506725e-6eae-4672-b0b5-f2021ec5fc6a None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:49.002917 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:49.062396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847f483-8f98-43b1-ae6e-fc1a07ebb068 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:49.097261 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506725e-6eae-4672-b0b5-f2021ec5fc6a None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:49.101716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea376f-3c54-46d4-ae5a-9e9b24774017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:49.102698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea376f-3c54-46d4-ae5a-9e9b24774017 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:49.102980 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea376f-3c54-46d4-ae5a-9e9b24774017 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:49.103254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea376f-3c54-46d4-ae5a-9e9b24774017 None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:49.104491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea376f-3c54-46d4-ae5a-9e9b24774017 None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:49.106058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea376f-3c54-46d4-ae5a-9e9b24774017 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:49.108396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 59/120] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:09:48 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:49.114057 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea376f-3c54-46d4-ae5a-9e9b24774017 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:49.125974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:49.141361 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85514572-b16f-4801-954d-47919be54216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:49.142122 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85514572-b16f-4801-954d-47919be54216 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:49.142407 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85514572-b16f-4801-954d-47919be54216 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:49.142740 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85514572-b16f-4801-954d-47919be54216 None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:49.145397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85514572-b16f-4801-954d-47919be54216 None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:49.147230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85514572-b16f-4801-954d-47919be54216 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:49.154526 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 62/121] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:09:48 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:49.155696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526834e6-a857-43c4-90f2-23be538a204c None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:49.155979 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526834e6-a857-43c4-90f2-23be538a204c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:49.156284 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526834e6-a857-43c4-90f2-23be538a204c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:49.176130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-526834e6-a857-43c4-90f2-23be538a204c None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:49.177383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-526834e6-a857-43c4-90f2-23be538a204c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:49.229000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea376f-3c54-46d4-ae5a-9e9b24774017 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:49.263616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 63/122] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:49.263616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526834e6-a857-43c4-90f2-23be538a204c None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:49.273639 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a0f2f-dd18-4780-98f3-d22a7f3c5dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:49.274403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a0f2f-dd18-4780-98f3-d22a7f3c5dbd None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:49.274787 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a0f2f-dd18-4780-98f3-d22a7f3c5dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:49.275173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a0f2f-dd18-4780-98f3-d22a7f3c5dbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:49.285532 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:49.368506 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a0f2f-dd18-4780-98f3-d22a7f3c5dbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:49.370025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a0f2f-dd18-4780-98f3-d22a7f3c5dbd None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:49.375109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526834e6-a857-43c4-90f2-23be538a204c None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:49.380169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:49.422196 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b2f05-1cd7-4feb-b914-3d9924266fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:49.423313 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b2f05-1cd7-4feb-b914-3d9924266fcc None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:49.423748 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b2f05-1cd7-4feb-b914-3d9924266fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:49.424133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b2f05-1cd7-4feb-b914-3d9924266fcc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:49.437288 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b2f05-1cd7-4feb-b914-3d9924266fcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:49.442308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-040b2f05-1cd7-4feb-b914-3d9924266fcc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:09:49.442308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:49.442308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:49.442308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:49.442308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:49.442308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:49.442308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:49.442308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:49.442308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:49.442308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:49.442308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:49.442308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:49.442308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:49.442308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:49.442308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:49.442308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:49.443419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:49.443419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:49.443419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:49.443419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:49.443419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:49.443419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:49.443419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:49.443419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:49.443419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:49.443419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:49.443419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:49.443419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:49.443419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:49.443419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:49.443419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:49.443419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:49.444364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:49.444364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:49.444364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:49.444364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:49.444364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:49.444364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:49.444364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:49.444364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:49.444364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:49.444364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:49.444364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:09:49.444364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:09:49.448274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 64/123] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:09:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:49.449133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b2f05-1cd7-4feb-b914-3d9924266fcc None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:49.501701 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 60/124] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:09:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:09:49.502780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a0f2f-dd18-4780-98f3-d22a7f3c5dbd None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:49.546752 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:49.570960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b2f05-1cd7-4feb-b914-3d9924266fcc None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:49.616731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a0f2f-dd18-4780-98f3-d22a7f3c5dbd None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:49.617076 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaa78d-1382-4f60-843a-a421b0bdf11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:49.617799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa78d-1382-4f60-843a-a421b0bdf11a None admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:49.618083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa78d-1382-4f60-843a-a421b0bdf11a None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:49.618363 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa78d-1382-4f60-843a-a421b0bdf11a None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17/users/a0dfd1dc38b741ebadff99a9c1fa4e99/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:49.620372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abaa78d-1382-4f60-843a-a421b0bdf11a None admin] RBAC: Authorizing `identity:create_grant(project_id=ab228684d11649dfa97b4acebd727e17, user_id=a0dfd1dc38b741ebadff99a9c1fa4e99, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:49.621876 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abaa78d-1382-4f60-843a-a421b0bdf11a None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:49.628141 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:49.639108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 65/125] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:09:49 2022] PUT /identity/v3/projects/ab228684d11649dfa97b4acebd727e17/users/a0dfd1dc38b741ebadff99a9c1fa4e99/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:09:49.640238 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaa78d-1382-4f60-843a-a421b0bdf11a None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:49.668337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ab80a-a20c-4c91-bcb8-269961341d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:49.669153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ab80a-a20c-4c91-bcb8-269961341d9e None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:49.669503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ab80a-a20c-4c91-bcb8-269961341d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:49.669902 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ab80a-a20c-4c91-bcb8-269961341d9e None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17/users/a0dfd1dc38b741ebadff99a9c1fa4e99/roles/7ae6dccec1244987a5bc7bf5c3991703` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:49.672628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ab80a-a20c-4c91-bcb8-269961341d9e None admin] RBAC: Authorizing `identity:create_grant(project_id=ab228684d11649dfa97b4acebd727e17, user_id=a0dfd1dc38b741ebadff99a9c1fa4e99, role_id=7ae6dccec1244987a5bc7bf5c3991703)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:49.674465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ab80a-a20c-4c91-bcb8-269961341d9e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:49.695471 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 61/126] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:09:49 2022] PUT /identity/v3/projects/ab228684d11649dfa97b4acebd727e17/users/a0dfd1dc38b741ebadff99a9c1fa4e99/roles/7ae6dccec1244987a5bc7bf5c3991703 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:09:49.702422 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ab80a-a20c-4c91-bcb8-269961341d9e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:49.800607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaa78d-1382-4f60-843a-a421b0bdf11a None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:49.849194 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd956f3e-d752-4d6b-b8a8-86638600a298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:49.850266 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd956f3e-d752-4d6b-b8a8-86638600a298 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:49.850553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd956f3e-d752-4d6b-b8a8-86638600a298 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:49.850825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd956f3e-d752-4d6b-b8a8-86638600a298 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:49.859702 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:49.871518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd956f3e-d752-4d6b-b8a8-86638600a298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:49.886252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-cd956f3e-d752-4d6b-b8a8-86638600a298 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:09:49.886252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:49.886252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:49.886252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:49.886252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:49.886252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:49.886252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:49.886252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:49.886252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:49.886252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:49.886252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:49.886252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:49.886252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:49.886252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:49.886252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:49.886252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:49.887241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:49.887241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:49.887241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:49.887241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:49.887241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:49.887241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:49.887241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:49.887241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:49.887241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:49.887241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:49.887241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:49.887241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:49.887241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:49.887241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:49.887241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:49.887241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:49.888803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:49.888803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:49.888803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:49.888803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:49.888803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:49.888803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:49.888803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:49.888803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:49.888803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:49.888803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:49.888803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:09:49.888803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:09:49.888803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 66/127] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:09:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:49.888803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd956f3e-d752-4d6b-b8a8-86638600a298 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:49.956113 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ab80a-a20c-4c91-bcb8-269961341d9e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:50.028994 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a90f43-bf29-4fa9-ab14-3ff4907e6331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:50.029420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a90f43-bf29-4fa9-ab14-3ff4907e6331 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:50.029676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a90f43-bf29-4fa9-ab14-3ff4907e6331 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:50.029996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a90f43-bf29-4fa9-ab14-3ff4907e6331 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:50.031014 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a90f43-bf29-4fa9-ab14-3ff4907e6331 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:50.031386 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a90f43-bf29-4fa9-ab14-3ff4907e6331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:50.032939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a90f43-bf29-4fa9-ab14-3ff4907e6331 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:50.043857 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 62/128] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:09:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:50.044603 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a90f43-bf29-4fa9-ab14-3ff4907e6331 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:50.061322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd956f3e-d752-4d6b-b8a8-86638600a298 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:50.091085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:50.129508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62b790-4341-4fa0-887d-c92e654b7111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:50.130847 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62b790-4341-4fa0-887d-c92e654b7111 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:50.131811 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62b790-4341-4fa0-887d-c92e654b7111 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:50.132838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62b790-4341-4fa0-887d-c92e654b7111 None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:50.135027 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62b790-4341-4fa0-887d-c92e654b7111 None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:50.137231 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62b790-4341-4fa0-887d-c92e654b7111 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:50.140251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 67/129] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:09:49 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:50.146121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62b790-4341-4fa0-887d-c92e654b7111 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:50.198262 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a90f43-bf29-4fa9-ab14-3ff4907e6331 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:50.227272 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:50.255197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58800188-a266-4e6a-89e6-84e98f8fdf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:50.256587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58800188-a266-4e6a-89e6-84e98f8fdf1e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:50.257848 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58800188-a266-4e6a-89e6-84e98f8fdf1e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:50.258400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58800188-a266-4e6a-89e6-84e98f8fdf1e None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:50.259798 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58800188-a266-4e6a-89e6-84e98f8fdf1e None admin] RBAC: Adding query filter params (name=member) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:50.260585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58800188-a266-4e6a-89e6-84e98f8fdf1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:50.263015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58800188-a266-4e6a-89e6-84e98f8fdf1e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:50.271405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62b790-4341-4fa0-887d-c92e654b7111 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:50.276588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 63/130] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:09:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:50.278391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece22f0-1af5-42fe-adcc-842f97788817 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:50.278912 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece22f0-1af5-42fe-adcc-842f97788817 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:50.280128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece22f0-1af5-42fe-adcc-842f97788817 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:50.281876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 64/131] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:50.283541 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece22f0-1af5-42fe-adcc-842f97788817 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:50.328409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e8890-681c-4c08-9008-9b1526b6a491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:50.329855 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e8890-681c-4c08-9008-9b1526b6a491 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:50.330468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e8890-681c-4c08-9008-9b1526b6a491 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:50.330970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e8890-681c-4c08-9008-9b1526b6a491 None admin] PATH_INFO: `/v3/users/1b4b6acdd88e421a86924fd2c155794f` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:50.333126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e8890-681c-4c08-9008-9b1526b6a491 None admin] RBAC: Authorizing `identity:get_user(user_id=1b4b6acdd88e421a86924fd2c155794f)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:50.335597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e8890-681c-4c08-9008-9b1526b6a491 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:50.337945 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 68/132] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:50 2022] GET /identity/v3/users/1b4b6acdd88e421a86924fd2c155794f => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:50.343961 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e8890-681c-4c08-9008-9b1526b6a491 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:50.417702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:50.435567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece22f0-1af5-42fe-adcc-842f97788817 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:50.484344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e8890-681c-4c08-9008-9b1526b6a491 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:50.485430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76e759-75b1-435e-82c0-f0556a1176eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:50.486266 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76e759-75b1-435e-82c0-f0556a1176eb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:50.486679 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76e759-75b1-435e-82c0-f0556a1176eb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:50.487138 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76e759-75b1-435e-82c0-f0556a1176eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:50.488174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76e759-75b1-435e-82c0-f0556a1176eb None admin] RBAC: Adding query filter params (role.id=24102e3afb81458ca8d594a2b25b31b2, scope.project.id=ab228684d11649dfa97b4acebd727e17, user.id=f526a3c1fd564b2d9d3bec4abdac8135) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:50.488640 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76e759-75b1-435e-82c0-f0556a1176eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:50.490300 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76e759-75b1-435e-82c0-f0556a1176eb None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:50.497374 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 65/133] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:09:50 2022] GET /identity/v3/role_assignments?user.id=f526a3c1fd564b2d9d3bec4abdac8135&scope.project.id=ab228684d11649dfa97b4acebd727e17&role.id=24102e3afb81458ca8d594a2b25b31b2 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:50.499175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92bf8d-b6b4-4206-9291-3710a3e4f9d7 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:50.499615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92bf8d-b6b4-4206-9291-3710a3e4f9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:50.500037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92bf8d-b6b4-4206-9291-3710a3e4f9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:50.536469 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ea92bf8d-b6b4-4206-9291-3710a3e4f9d7 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:50.538171 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ea92bf8d-b6b4-4206-9291-3710a3e4f9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:50.539533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e458ec-6a9d-4570-b8e6-1d2590057358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:50.540294 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e458ec-6a9d-4570-b8e6-1d2590057358 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:50.540645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e458ec-6a9d-4570-b8e6-1d2590057358 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:50.540989 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e458ec-6a9d-4570-b8e6-1d2590057358 None admin] PATH_INFO: `/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:50.542312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e458ec-6a9d-4570-b8e6-1d2590057358 None admin] RBAC: Authorizing `identity:get_user(user_id=a0dfd1dc38b741ebadff99a9c1fa4e99)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:50.545635 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e458ec-6a9d-4570-b8e6-1d2590057358 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:50.550387 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 69/134] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:50 2022] GET /identity/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99 => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:50.556786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e458ec-6a9d-4570-b8e6-1d2590057358 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:50.562499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:50.671583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 66/135] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:50.673010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92bf8d-b6b4-4206-9291-3710a3e4f9d7 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:50.783117 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e458ec-6a9d-4570-b8e6-1d2590057358 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:50.817885 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:50.824318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a93082-cb42-41f1-8b2e-d83e878f64a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:50.825058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a93082-cb42-41f1-8b2e-d83e878f64a1 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:50.825348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a93082-cb42-41f1-8b2e-d83e878f64a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:50.825633 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a93082-cb42-41f1-8b2e-d83e878f64a1 None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:50.831368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a93082-cb42-41f1-8b2e-d83e878f64a1 None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:50.832962 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a93082-cb42-41f1-8b2e-d83e878f64a1 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:50.837901 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92bf8d-b6b4-4206-9291-3710a3e4f9d7 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:50.838846 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 70/136] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:09:50 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:50.839691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a93082-cb42-41f1-8b2e-d83e878f64a1 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:50.892861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a282c-2312-4a4f-bde0-87077f223f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:50.893604 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a282c-2312-4a4f-bde0-87077f223f76 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:50.894024 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a282c-2312-4a4f-bde0-87077f223f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:50.894421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a282c-2312-4a4f-bde0-87077f223f76 None admin] PATH_INFO: `/v3/projects/541d4aab26d64344bf63c9afdab2dc6f` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:50.902841 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a282c-2312-4a4f-bde0-87077f223f76 None admin] RBAC: Authorizing `identity:get_project(project_id=541d4aab26d64344bf63c9afdab2dc6f)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:50.904584 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a282c-2312-4a4f-bde0-87077f223f76 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:50.906904 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 67/137] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:09:50 2022] GET /identity/v3/projects/541d4aab26d64344bf63c9afdab2dc6f => generated 312 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:50.907761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a282c-2312-4a4f-bde0-87077f223f76 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:50.953835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:51.002937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a93082-cb42-41f1-8b2e-d83e878f64a1 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:51.060986 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26c0d6-7dbb-4fdd-ab31-b7943e0739e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:51.061584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26c0d6-7dbb-4fdd-ab31-b7943e0739e2 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:51.061907 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26c0d6-7dbb-4fdd-ab31-b7943e0739e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:51.062178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26c0d6-7dbb-4fdd-ab31-b7943e0739e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:51.064453 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a282c-2312-4a4f-bde0-87077f223f76 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:51.066681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26c0d6-7dbb-4fdd-ab31-b7943e0739e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:51.068327 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26c0d6-7dbb-4fdd-ab31-b7943e0739e2 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:51.070073 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 71/138] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:09:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:51.070836 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26c0d6-7dbb-4fdd-ab31-b7943e0739e2 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:51.136728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5f8a4-d689-41f1-9531-653c0c1a2b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:51.137748 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5f8a4-d689-41f1-9531-653c0c1a2b55 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:51.137748 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5f8a4-d689-41f1-9531-653c0c1a2b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:51.138027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5f8a4-d689-41f1-9531-653c0c1a2b55 None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17/users/1b4b6acdd88e421a86924fd2c155794f/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:51.140170 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5f8a4-d689-41f1-9531-653c0c1a2b55 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab228684d11649dfa97b4acebd727e17, user_id=1b4b6acdd88e421a86924fd2c155794f, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:51.141650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5f8a4-d689-41f1-9531-653c0c1a2b55 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:51.185685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 68/139] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:09:50 2022] PUT /identity/v3/projects/ab228684d11649dfa97b4acebd727e17/users/1b4b6acdd88e421a86924fd2c155794f/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:09:51.190174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5f8a4-d689-41f1-9531-653c0c1a2b55 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:51.207915 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26c0d6-7dbb-4fdd-ab31-b7943e0739e2 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:51.222459 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:51.304259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2877571e-f4b1-4e9e-bc26-f35dcc55519d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:51.304259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877571e-f4b1-4e9e-bc26-f35dcc55519d None admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:51.304259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877571e-f4b1-4e9e-bc26-f35dcc55519d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:51.304259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2877571e-f4b1-4e9e-bc26-f35dcc55519d None admin] PATH_INFO: `/v3/projects/541d4aab26d64344bf63c9afdab2dc6f/users/a0dfd1dc38b741ebadff99a9c1fa4e99/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:51.311172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877571e-f4b1-4e9e-bc26-f35dcc55519d None admin] RBAC: Authorizing `identity:create_grant(project_id=541d4aab26d64344bf63c9afdab2dc6f, user_id=a0dfd1dc38b741ebadff99a9c1fa4e99, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:51.315126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2877571e-f4b1-4e9e-bc26-f35dcc55519d None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:51.334582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:51.349393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 72/140] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:09:51 2022] PUT /identity/v3/projects/541d4aab26d64344bf63c9afdab2dc6f/users/a0dfd1dc38b741ebadff99a9c1fa4e99/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:09:51.413753 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5f8a4-d689-41f1-9531-653c0c1a2b55 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:51.476917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381195a-5437-4dd5-a41f-f139aa5449dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:51.477580 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381195a-5437-4dd5-a41f-f139aa5449dc None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:51.477892 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381195a-5437-4dd5-a41f-f139aa5449dc None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:51.478177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381195a-5437-4dd5-a41f-f139aa5449dc None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:51.479112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381195a-5437-4dd5-a41f-f139aa5449dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:51.480700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381195a-5437-4dd5-a41f-f139aa5449dc None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:51.519112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 69/141] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:09:51 2022] POST /identity/v3/users => generated 283 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:09:52.185818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85479c41-c59e-4b5d-846b-38adf9b4c797 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:52.186346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85479c41-c59e-4b5d-846b-38adf9b4c797 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:52.186346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85479c41-c59e-4b5d-846b-38adf9b4c797 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:52.187661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 73/142] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:52.195602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d18cf-eccd-4ac0-900b-e5d4f9c905ee None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:52.196068 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d18cf-eccd-4ac0-900b-e5d4f9c905ee None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:52.196149 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d18cf-eccd-4ac0-900b-e5d4f9c905ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:52.217105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-201d18cf-eccd-4ac0-900b-e5d4f9c905ee None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:52.218353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-201d18cf-eccd-4ac0-900b-e5d4f9c905ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:52.302800 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:52.312836 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 70/143] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:52.347162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85479c41-c59e-4b5d-846b-38adf9b4c797 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:52.440389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7085e88-8ce8-4095-93aa-c687b4cead24 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:52.440684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7085e88-8ce8-4095-93aa-c687b4cead24 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:52.440862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7085e88-8ce8-4095-93aa-c687b4cead24 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:52.442536 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 71/144] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:52.455303 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aab8b-ce06-4e16-87d1-f9d631d59ab8 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:52.455563 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aab8b-ce06-4e16-87d1-f9d631d59ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:52.455844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150aab8b-ce06-4e16-87d1-f9d631d59ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:52.486471 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-150aab8b-ce06-4e16-87d1-f9d631d59ab8 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:52.487686 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-150aab8b-ce06-4e16-87d1-f9d631d59ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:52.496414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85479c41-c59e-4b5d-846b-38adf9b4c797 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:52.568083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377079cd-1cc2-4e44-87ef-eb635bb477c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:52.569091 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377079cd-1cc2-4e44-87ef-eb635bb477c8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:52.569513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377079cd-1cc2-4e44-87ef-eb635bb477c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:52.570002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377079cd-1cc2-4e44-87ef-eb635bb477c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:52.599920 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377079cd-1cc2-4e44-87ef-eb635bb477c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:52.606047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-377079cd-1cc2-4e44-87ef-eb635bb477c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:09:52.606047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:52.606047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:52.606047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:52.606047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:52.606047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:52.606047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:52.606047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:52.606047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:52.606047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:52.606047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:52.606047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:52.606047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:52.606047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:52.606047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:52.606047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:52.607093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:52.607093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:52.607093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:52.607093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:52.607093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:52.607093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:52.607093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:52.607093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:52.607093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:52.607093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:52.607093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:52.607093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:52.607093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:52.607093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:52.607093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:52.607093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:52.608048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:52.608048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:52.608048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:52.608048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:52.608048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:52.608048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:52.608048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:52.608048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:52.608048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:52.608048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:52.608048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:09:52.608048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:09:52.610457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 74/145] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:09:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:52.618371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377079cd-1cc2-4e44-87ef-eb635bb477c8 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:52.640981 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 72/146] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:52.671301 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150aab8b-ce06-4e16-87d1-f9d631d59ab8 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:52.736152 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:52.838047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377079cd-1cc2-4e44-87ef-eb635bb477c8 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:52.854015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150aab8b-ce06-4e16-87d1-f9d631d59ab8 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:52.903012 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:52.911652 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af418541-1cc0-42eb-bc2b-345353a5adde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:52.912616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af418541-1cc0-42eb-bc2b-345353a5adde None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:52.912908 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af418541-1cc0-42eb-bc2b-345353a5adde None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:52.913177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af418541-1cc0-42eb-bc2b-345353a5adde None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:52.929148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af418541-1cc0-42eb-bc2b-345353a5adde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:52.931861 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197090b9-a1d7-4b80-9f9a-1cead15259e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:52.932524 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197090b9-a1d7-4b80-9f9a-1cead15259e6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:52.932821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197090b9-a1d7-4b80-9f9a-1cead15259e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:52.933102 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197090b9-a1d7-4b80-9f9a-1cead15259e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:52.933874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197090b9-a1d7-4b80-9f9a-1cead15259e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:52.934183 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197090b9-a1d7-4b80-9f9a-1cead15259e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:52.935679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197090b9-a1d7-4b80-9f9a-1cead15259e6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:52.937053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-af418541-1cc0-42eb-bc2b-345353a5adde None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:09:52.937053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:52.937053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:52.937053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:52.937053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:52.937053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:52.937053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:52.937053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:52.937053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:52.937053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:52.937053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:52.937053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:52.937053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:52.937053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:52.937053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:52.937053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:52.938546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:52.938546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:52.938546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:52.938546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:52.938546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:52.938546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:52.938546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:52.938546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:52.938546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:52.938546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:52.938546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:52.938546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:52.938546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:52.938546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:52.938546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:52.938546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:52.941437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:52.941437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:52.941437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:52.941437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:52.941437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:52.941437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:52.941437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:52.941437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:52.941437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:52.941437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:52.941437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:09:52.941437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:09:52.941437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 73/147] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:09:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:09:52.945285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af418541-1cc0-42eb-bc2b-345353a5adde None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:52.959168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 75/148] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:09:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:52.972827 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197090b9-a1d7-4b80-9f9a-1cead15259e6 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:53.107544 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af418541-1cc0-42eb-bc2b-345353a5adde None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:53.160550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7265d1a-3da1-4a1d-aba7-cda16c35af3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:53.161567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7265d1a-3da1-4a1d-aba7-cda16c35af3d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:53.162088 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7265d1a-3da1-4a1d-aba7-cda16c35af3d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:53.162563 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7265d1a-3da1-4a1d-aba7-cda16c35af3d None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:53.163506 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7265d1a-3da1-4a1d-aba7-cda16c35af3d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:53.164034 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7265d1a-3da1-4a1d-aba7-cda16c35af3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:53.165700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7265d1a-3da1-4a1d-aba7-cda16c35af3d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:53.185414 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 74/149] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:09:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:53.193687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197090b9-a1d7-4b80-9f9a-1cead15259e6 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:53.215818 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7265d1a-3da1-4a1d-aba7-cda16c35af3d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:53.222576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:53.238219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8fe89-8240-406c-a518-e45363e5734e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:53.238964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8fe89-8240-406c-a518-e45363e5734e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:53.239293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8fe89-8240-406c-a518-e45363e5734e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:53.239572 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8fe89-8240-406c-a518-e45363e5734e None admin] PATH_INFO: `/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:53.240839 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8fe89-8240-406c-a518-e45363e5734e None admin] RBAC: Authorizing `identity:get_user(user_id=a0dfd1dc38b741ebadff99a9c1fa4e99)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:53.242358 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8fe89-8240-406c-a518-e45363e5734e None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:53.244134 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 76/150] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:52 2022] GET /identity/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99 => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:53.250030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8fe89-8240-406c-a518-e45363e5734e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:53.293031 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:53.349056 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7265d1a-3da1-4a1d-aba7-cda16c35af3d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:53.388937 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d6625-0d3d-415d-845e-66b1bcf40a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:53.390491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d6625-0d3d-415d-845e-66b1bcf40a16 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:53.391271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d6625-0d3d-415d-845e-66b1bcf40a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:53.391927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d6625-0d3d-415d-845e-66b1bcf40a16 None admin] PATH_INFO: `/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:53.394384 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d6625-0d3d-415d-845e-66b1bcf40a16 None admin] RBAC: Authorizing `identity:get_user(user_id=a0dfd1dc38b741ebadff99a9c1fa4e99)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:53.397242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d6625-0d3d-415d-845e-66b1bcf40a16 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:53.399486 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8fe89-8240-406c-a518-e45363e5734e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:53.401051 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 75/151] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:53 2022] GET /identity/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:09:53.409986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d6625-0d3d-415d-845e-66b1bcf40a16 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:53.438068 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b3fc5-ea39-4000-9705-77164dec3bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:53.438686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b3fc5-ea39-4000-9705-77164dec3bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:53.438956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b3fc5-ea39-4000-9705-77164dec3bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:53.439221 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b3fc5-ea39-4000-9705-77164dec3bc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:53.503906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:53.516270 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b3fc5-ea39-4000-9705-77164dec3bc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:53.520324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b3fc5-ea39-4000-9705-77164dec3bc3 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:53.623182 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d6625-0d3d-415d-845e-66b1bcf40a16 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:53.654355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 77/152] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:09:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:09:53.655859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:53.661818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b3fc5-ea39-4000-9705-77164dec3bc3 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:53.684625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94561828-b191-4cb4-a658-326a08989a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:53.685975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94561828-b191-4cb4-a658-326a08989a63 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:53.686496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94561828-b191-4cb4-a658-326a08989a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:53.686976 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94561828-b191-4cb4-a658-326a08989a63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:53.758976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:53.771110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b3fc5-ea39-4000-9705-77164dec3bc3 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:53.797146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94561828-b191-4cb4-a658-326a08989a63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:53.802854 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94561828-b191-4cb4-a658-326a08989a63 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:53.811304 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606c035-2b70-470d-8092-ba3d70c6a4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:53.813014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606c035-2b70-470d-8092-ba3d70c6a4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:53.813353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606c035-2b70-470d-8092-ba3d70c6a4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:53.813631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606c035-2b70-470d-8092-ba3d70c6a4e0 None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:53.815018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606c035-2b70-470d-8092-ba3d70c6a4e0 None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:53.816634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606c035-2b70-470d-8092-ba3d70c6a4e0 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:53.818871 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 78/153] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:09:53 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:09:53.820011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cff160-e51e-4a2a-be3f-2a3009d7d028 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:53.820297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cff160-e51e-4a2a-be3f-2a3009d7d028 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:53.820706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cff160-e51e-4a2a-be3f-2a3009d7d028 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:53.826159 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 79/154] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:53.828114 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cff160-e51e-4a2a-be3f-2a3009d7d028 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:53.948186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cff160-e51e-4a2a-be3f-2a3009d7d028 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:53.960827 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 76/155] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:09:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:09:53.962213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49dd57-5104-42ab-91a3-fdd9dcd8beb7 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:53.962496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49dd57-5104-42ab-91a3-fdd9dcd8beb7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:53.962769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49dd57-5104-42ab-91a3-fdd9dcd8beb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:53.987508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215fc907-215c-433f-9304-5a9273237dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:53.988584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fc907-215c-433f-9304-5a9273237dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:53.989058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fc907-215c-433f-9304-5a9273237dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:53.989552 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fc907-215c-433f-9304-5a9273237dd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:53.990619 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215fc907-215c-433f-9304-5a9273237dd1 None admin] RBAC: Adding query filter params (role.id=19db9fb1cb1541ee94673df232bd6952, scope.project.id=ab228684d11649dfa97b4acebd727e17, user.id=a0dfd1dc38b741ebadff99a9c1fa4e99) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:53.991157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215fc907-215c-433f-9304-5a9273237dd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:53.993767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215fc907-215c-433f-9304-5a9273237dd1 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:53.995391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-cb49dd57-5104-42ab-91a3-fdd9dcd8beb7 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:53.997241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cb49dd57-5104-42ab-91a3-fdd9dcd8beb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:54.005486 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 80/156] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:09:53 2022] GET /identity/v3/role_assignments?user.id=a0dfd1dc38b741ebadff99a9c1fa4e99&scope.project.id=ab228684d11649dfa97b4acebd727e17&role.id=19db9fb1cb1541ee94673df232bd6952 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:09:54.007198 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215fc907-215c-433f-9304-5a9273237dd1 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:54.042218 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:54.052231 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:54.133522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215fc907-215c-433f-9304-5a9273237dd1 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:54.173075 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61a347-40e6-4f85-9717-dd2b1a1b845a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:54.174290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61a347-40e6-4f85-9717-dd2b1a1b845a None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:54.174766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61a347-40e6-4f85-9717-dd2b1a1b845a None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:54.175299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61a347-40e6-4f85-9717-dd2b1a1b845a None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:54.176789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61a347-40e6-4f85-9717-dd2b1a1b845a None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:54.178580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61a347-40e6-4f85-9717-dd2b1a1b845a None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:54.180513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 81/157] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:09:54 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:54.185687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 77/158] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:54.187133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61a347-40e6-4f85-9717-dd2b1a1b845a None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:54.235712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49dd57-5104-42ab-91a3-fdd9dcd8beb7 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:54.302234 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:54.320054 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61a347-40e6-4f85-9717-dd2b1a1b845a None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:54.373505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ce9fb-f808-4257-84ec-422e4a6a92e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:54.374638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ce9fb-f808-4257-84ec-422e4a6a92e6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:54.375128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ce9fb-f808-4257-84ec-422e4a6a92e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:54.375612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ce9fb-f808-4257-84ec-422e4a6a92e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:54.376650 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ce9fb-f808-4257-84ec-422e4a6a92e6 None admin] RBAC: Adding query filter params (role.id=7ae6dccec1244987a5bc7bf5c3991703, scope.project.id=ab228684d11649dfa97b4acebd727e17, user.id=a0dfd1dc38b741ebadff99a9c1fa4e99) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:54.377156 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ce9fb-f808-4257-84ec-422e4a6a92e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:54.378983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ce9fb-f808-4257-84ec-422e4a6a92e6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:54.391027 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 82/159] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:09:54 2022] GET /identity/v3/role_assignments?user.id=a0dfd1dc38b741ebadff99a9c1fa4e99&scope.project.id=ab228684d11649dfa97b4acebd727e17&role.id=7ae6dccec1244987a5bc7bf5c3991703 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:54.403891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49dd57-5104-42ab-91a3-fdd9dcd8beb7 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:54.457465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33f9cb-40cc-4cf5-b679-f6ea34916b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:54.458169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33f9cb-40cc-4cf5-b679-f6ea34916b98 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:54.458449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33f9cb-40cc-4cf5-b679-f6ea34916b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:54.458722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33f9cb-40cc-4cf5-b679-f6ea34916b98 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:54.476213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc33f9cb-40cc-4cf5-b679-f6ea34916b98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:54.481285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-cc33f9cb-40cc-4cf5-b679-f6ea34916b98 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:09:54.481285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:54.481285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:54.481285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:54.481285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:54.481285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:54.481285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:54.481285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:54.481285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:54.481285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:54.481285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:54.481285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:54.481285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:54.481285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:54.481285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:54.481285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:54.482227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:54.482227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:54.482227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:54.482227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:54.482227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:54.482227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:54.482227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:54.482227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:54.482227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:54.482227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:54.482227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:54.482227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:54.482227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:54.482227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:54.482227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:54.482227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:54.483005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:54.483005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:54.483005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:54.483005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:54.483005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:54.483005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:54.483005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:54.483005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:54.483005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:54.483005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:54.483005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:09:54.483005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:09:54.483005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 78/160] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:09:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:09:54.489208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ce9fb-f808-4257-84ec-422e4a6a92e6 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:54.623120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ce9fb-f808-4257-84ec-422e4a6a92e6 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:54.662382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cd9f3-5352-4716-9b6b-c464e2ef5b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:54.663524 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cd9f3-5352-4716-9b6b-c464e2ef5b35 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:54.664012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cd9f3-5352-4716-9b6b-c464e2ef5b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:54.664497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cd9f3-5352-4716-9b6b-c464e2ef5b35 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:54.665441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cd9f3-5352-4716-9b6b-c464e2ef5b35 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:54.665979 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cd9f3-5352-4716-9b6b-c464e2ef5b35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:54.667611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cd9f3-5352-4716-9b6b-c464e2ef5b35 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:54.677689 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 83/161] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:09:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:54.683643 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33f9cb-40cc-4cf5-b679-f6ea34916b98 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:54.709287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:54.832666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33f9cb-40cc-4cf5-b679-f6ea34916b98 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:54.879241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de7062-65ce-4938-b079-20782c109a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:54.879344 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de7062-65ce-4938-b079-20782c109a02 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:54.879613 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de7062-65ce-4938-b079-20782c109a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:54.881900 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de7062-65ce-4938-b079-20782c109a02 None admin] PATH_INFO: `/v3/users/f526a3c1fd564b2d9d3bec4abdac8135` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:54.882602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de7062-65ce-4938-b079-20782c109a02 None admin] RBAC: Authorizing `identity:get_user(user_id=f526a3c1fd564b2d9d3bec4abdac8135)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:54.884095 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de7062-65ce-4938-b079-20782c109a02 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:54.885771 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 79/162] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:54 2022] GET /identity/v3/users/f526a3c1fd564b2d9d3bec4abdac8135 => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:54.892449 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cd9f3-5352-4716-9b6b-c464e2ef5b35 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:54.925807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:55.040252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cd9f3-5352-4716-9b6b-c464e2ef5b35 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:55.077965 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82fc98a-e3dc-4d7e-9549-fa0be120ff2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:55.079024 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fc98a-e3dc-4d7e-9549-fa0be120ff2b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:55.079499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fc98a-e3dc-4d7e-9549-fa0be120ff2b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:55.079964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fc98a-e3dc-4d7e-9549-fa0be120ff2b None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:55.081464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82fc98a-e3dc-4d7e-9549-fa0be120ff2b None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:55.083235 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82fc98a-e3dc-4d7e-9549-fa0be120ff2b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:55.085182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 84/163] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:09:54 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:55.091105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de7062-65ce-4938-b079-20782c109a02 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:55.181158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd85d82-1bb1-42f8-ab8c-e87bb418f968 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:55.181869 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd85d82-1bb1-42f8-ab8c-e87bb418f968 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:55.182406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd85d82-1bb1-42f8-ab8c-e87bb418f968 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:55.184002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 85/164] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:55.189562 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f09aa5-1c6e-4452-b5e6-86daf8172b04 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:55.190155 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f09aa5-1c6e-4452-b5e6-86daf8172b04 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:55.190630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:55.192000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f09aa5-1c6e-4452-b5e6-86daf8172b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:55.214057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-a6f09aa5-1c6e-4452-b5e6-86daf8172b04 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:55.215632 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-a6f09aa5-1c6e-4452-b5e6-86daf8172b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:55.225136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de7062-65ce-4938-b079-20782c109a02 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:55.279543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11caa231-40e4-4ae9-a97a-3025013d4960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:55.280338 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11caa231-40e4-4ae9-a97a-3025013d4960 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:55.280567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11caa231-40e4-4ae9-a97a-3025013d4960 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:55.280846 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11caa231-40e4-4ae9-a97a-3025013d4960 None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17/users/f526a3c1fd564b2d9d3bec4abdac8135/roles/24102e3afb81458ca8d594a2b25b31b2` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:55.287296 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11caa231-40e4-4ae9-a97a-3025013d4960 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab228684d11649dfa97b4acebd727e17, user_id=f526a3c1fd564b2d9d3bec4abdac8135, role_id=24102e3afb81458ca8d594a2b25b31b2)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:55.289038 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11caa231-40e4-4ae9-a97a-3025013d4960 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:55.295600 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 86/165] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:09:55.316475 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 80/166] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:09:55 2022] PUT /identity/v3/projects/ab228684d11649dfa97b4acebd727e17/users/f526a3c1fd564b2d9d3bec4abdac8135/roles/24102e3afb81458ca8d594a2b25b31b2 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:09:55.325822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f09aa5-1c6e-4452-b5e6-86daf8172b04 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:55.355727 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:55.445283 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f09aa5-1c6e-4452-b5e6-86daf8172b04 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:55.487449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c00b1-d291-4d81-bbf0-46d5021c93ef None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:55.487934 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c00b1-d291-4d81-bbf0-46d5021c93ef None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:55.488080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c00b1-d291-4d81-bbf0-46d5021c93ef None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:55.489548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 81/167] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:55.495521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb963f6b-f66d-4bc4-ad7c-fb3a3dc0a8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:55.497481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb963f6b-f66d-4bc4-ad7c-fb3a3dc0a8ee None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:55.498437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb963f6b-f66d-4bc4-ad7c-fb3a3dc0a8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:55.499228 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb963f6b-f66d-4bc4-ad7c-fb3a3dc0a8ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:55.511405 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43f19a-2679-491f-9f0e-bb19cc5ceed3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:55.511673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43f19a-2679-491f-9f0e-bb19cc5ceed3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:55.512085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43f19a-2679-491f-9f0e-bb19cc5ceed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:55.522244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb963f6b-f66d-4bc4-ad7c-fb3a3dc0a8ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:55.532441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-bb963f6b-f66d-4bc4-ad7c-fb3a3dc0a8ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:09:55.532441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:55.532441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:55.532441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:55.532441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:55.532441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:55.532441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:55.532441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:55.532441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:55.532441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:55.532441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:55.532441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:55.532441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:55.532441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:55.532441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:55.532441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:55.534371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:55.534371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:55.534371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:55.534371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:55.534371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:55.534371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:55.534371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:55.534371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:55.534371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:55.534371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:55.534371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:55.534371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:55.534371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:55.534371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:55.534371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:55.534371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:55.535894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:55.535894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:55.535894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:55.535894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:55.535894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:55.535894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:55.535894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:55.535894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:55.535894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:55.535894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:55.535894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:09:55.535894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:09:55.540083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 87/168] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:09:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:55.547554 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb963f6b-f66d-4bc4-ad7c-fb3a3dc0a8ee None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:55.563030 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5e43f19a-2679-491f-9f0e-bb19cc5ceed3 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:55.564278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5e43f19a-2679-491f-9f0e-bb19cc5ceed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:55.651007 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:55.680033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb963f6b-f66d-4bc4-ad7c-fb3a3dc0a8ee None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:55.691817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 82/169] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:55.716071 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43f19a-2679-491f-9f0e-bb19cc5ceed3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:55.719572 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821355ab-ae15-4eb4-b340-be89ff03eb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:55.720577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821355ab-ae15-4eb4-b340-be89ff03eb84 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:55.721055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821355ab-ae15-4eb4-b340-be89ff03eb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:55.721517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821355ab-ae15-4eb4-b340-be89ff03eb84 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:55.722558 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821355ab-ae15-4eb4-b340-be89ff03eb84 None admin] RBAC: Adding query filter params (name=member) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:55.723141 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821355ab-ae15-4eb4-b340-be89ff03eb84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:55.724994 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821355ab-ae15-4eb4-b340-be89ff03eb84 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:55.735747 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 88/170] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:09:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:09:55.741800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821355ab-ae15-4eb4-b340-be89ff03eb84 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:55.796006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:55.832305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43f19a-2679-491f-9f0e-bb19cc5ceed3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:55.868065 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821355ab-ae15-4eb4-b340-be89ff03eb84 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:55.880982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955fabe-81be-4b9a-8c42-30869d38ab0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:55.882061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955fabe-81be-4b9a-8c42-30869d38ab0c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:55.882544 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955fabe-81be-4b9a-8c42-30869d38ab0c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:55.883060 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955fabe-81be-4b9a-8c42-30869d38ab0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:55.895735 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d955fabe-81be-4b9a-8c42-30869d38ab0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:55.901583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d955fabe-81be-4b9a-8c42-30869d38ab0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:09:55.901583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:55.901583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:55.901583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:55.901583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:55.901583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:55.901583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:55.901583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:55.901583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:55.901583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:55.901583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:55.901583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:55.901583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:55.901583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:55.901583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:55.901583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:55.902547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:55.902547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:55.902547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:55.902547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:55.902547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:55.902547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:55.902547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:55.902547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:55.902547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:55.902547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:55.902547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:55.902547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:55.902547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:55.902547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:55.902547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:55.902547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:55.903446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:55.903446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:55.903446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:55.903446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:55.903446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:55.903446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:55.903446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:55.903446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:55.903446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:55.903446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:55.903446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:09:55.903446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:09:55.906246 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 83/171] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:09:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:09:55.911839 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13355c-957e-442b-820d-1191edb32d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:55.912519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13355c-957e-442b-820d-1191edb32d3e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:55.912798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13355c-957e-442b-820d-1191edb32d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:55.913072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13355c-957e-442b-820d-1191edb32d3e None admin] PATH_INFO: `/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:55.915719 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13355c-957e-442b-820d-1191edb32d3e None admin] RBAC: Authorizing `identity:get_user(user_id=a0dfd1dc38b741ebadff99a9c1fa4e99)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:55.916506 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955fabe-81be-4b9a-8c42-30869d38ab0c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:55.917207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13355c-957e-442b-820d-1191edb32d3e None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:55.919167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 89/172] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:55 2022] GET /identity/v3/users/a0dfd1dc38b741ebadff99a9c1fa4e99 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:09:55.926969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13355c-957e-442b-820d-1191edb32d3e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:55.933210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:56.046720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d955fabe-81be-4b9a-8c42-30869d38ab0c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:56.064497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13355c-957e-442b-820d-1191edb32d3e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:56.076737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:56.085349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666cec3-5140-4009-99f7-45ac6c821aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:56.086032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666cec3-5140-4009-99f7-45ac6c821aae None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:56.086314 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666cec3-5140-4009-99f7-45ac6c821aae None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:56.086589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666cec3-5140-4009-99f7-45ac6c821aae None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:56.087330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666cec3-5140-4009-99f7-45ac6c821aae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:56.087634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666cec3-5140-4009-99f7-45ac6c821aae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:56.089055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666cec3-5140-4009-99f7-45ac6c821aae None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:56.099551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 84/173] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:09:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:56.105179 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666cec3-5140-4009-99f7-45ac6c821aae None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:56.106719 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee8fb2-9bb8-4e0a-9b55-37d0768e5cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:56.107689 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee8fb2-9bb8-4e0a-9b55-37d0768e5cba None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:56.108343 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee8fb2-9bb8-4e0a-9b55-37d0768e5cba None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:56.109468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee8fb2-9bb8-4e0a-9b55-37d0768e5cba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:56.187323 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee8fb2-9bb8-4e0a-9b55-37d0768e5cba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:56.189180 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee8fb2-9bb8-4e0a-9b55-37d0768e5cba None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:56.191802 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:56.222284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666cec3-5140-4009-99f7-45ac6c821aae None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:56.260719 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87283e9-309d-41b6-b381-90001df81e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:56.261628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87283e9-309d-41b6-b381-90001df81e26 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:56.261856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87283e9-309d-41b6-b381-90001df81e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:56.262046 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87283e9-309d-41b6-b381-90001df81e26 None admin] PATH_INFO: `/v3/users/1b4b6acdd88e421a86924fd2c155794f` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:56.263298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87283e9-309d-41b6-b381-90001df81e26 None admin] RBAC: Authorizing `identity:get_user(user_id=1b4b6acdd88e421a86924fd2c155794f)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:56.265126 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87283e9-309d-41b6-b381-90001df81e26 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:56.267233 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 85/174] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:56 2022] GET /identity/v3/users/1b4b6acdd88e421a86924fd2c155794f => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:56.273236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87283e9-309d-41b6-b381-90001df81e26 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:56.309237 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 90/175] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:09:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:09:56.315485 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee8fb2-9bb8-4e0a-9b55-37d0768e5cba None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:56.362184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:56.390323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87283e9-309d-41b6-b381-90001df81e26 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:56.450346 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f25187-4a62-43d9-a990-b8cfe05557da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:56.451477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f25187-4a62-43d9-a990-b8cfe05557da None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:56.451757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f25187-4a62-43d9-a990-b8cfe05557da None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:56.452030 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f25187-4a62-43d9-a990-b8cfe05557da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:56.463643 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee8fb2-9bb8-4e0a-9b55-37d0768e5cba None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:56.467179 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:56.537992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874941f7-d363-4428-9a4b-36e60a9fc1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:56.539045 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874941f7-d363-4428-9a4b-36e60a9fc1ad None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:56.539745 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874941f7-d363-4428-9a4b-36e60a9fc1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:56.541273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874941f7-d363-4428-9a4b-36e60a9fc1ad None admin] PATH_INFO: `/v3/projects/541d4aab26d64344bf63c9afdab2dc6f` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:56.542960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874941f7-d363-4428-9a4b-36e60a9fc1ad None admin] RBAC: Authorizing `identity:get_project(project_id=541d4aab26d64344bf63c9afdab2dc6f)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:56.545804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874941f7-d363-4428-9a4b-36e60a9fc1ad None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:56.548101 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 91/176] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:09:56 2022] GET /identity/v3/projects/541d4aab26d64344bf63c9afdab2dc6f => generated 312 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:09:56.555391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055889f9-ad6e-4939-8f80-d3e1fe86aa78 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:56.557483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055889f9-ad6e-4939-8f80-d3e1fe86aa78 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:56.558786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055889f9-ad6e-4939-8f80-d3e1fe86aa78 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:56.561375 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 92/177] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:56 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:56.564728 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055889f9-ad6e-4939-8f80-d3e1fe86aa78 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:56.580997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f25187-4a62-43d9-a990-b8cfe05557da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:56.582922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f25187-4a62-43d9-a990-b8cfe05557da None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:56.704900 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:56.781481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055889f9-ad6e-4939-8f80-d3e1fe86aa78 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:56.801176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 86/178] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:09:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:09:56.802854 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e87d29-7e57-49af-b5dd-186befa94b24 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:56.805245 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e87d29-7e57-49af-b5dd-186befa94b24 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:56.805753 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e87d29-7e57-49af-b5dd-186befa94b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:56.847646 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-88e87d29-7e57-49af-b5dd-186befa94b24 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:56.851517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-88e87d29-7e57-49af-b5dd-186befa94b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:56.858474 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e220e-a4eb-4b25-9bcf-87a720d018c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:56.860000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e220e-a4eb-4b25-9bcf-87a720d018c8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:56.860674 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e220e-a4eb-4b25-9bcf-87a720d018c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:56.862111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e220e-a4eb-4b25-9bcf-87a720d018c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:56.863277 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e220e-a4eb-4b25-9bcf-87a720d018c8 None admin] RBAC: Adding query filter params (role.id=19db9fb1cb1541ee94673df232bd6952, scope.project.id=541d4aab26d64344bf63c9afdab2dc6f, user.id=a0dfd1dc38b741ebadff99a9c1fa4e99) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:56.868469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e220e-a4eb-4b25-9bcf-87a720d018c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:56.870354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9e220e-a4eb-4b25-9bcf-87a720d018c8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:56.880455 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 93/179] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:09:56 2022] GET /identity/v3/role_assignments?user.id=a0dfd1dc38b741ebadff99a9c1fa4e99&scope.project.id=541d4aab26d64344bf63c9afdab2dc6f&role.id=19db9fb1cb1541ee94673df232bd6952 => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:56.890307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e220e-a4eb-4b25-9bcf-87a720d018c8 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:56.958069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:56.984159 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 87/180] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:56.985708 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45396fb-85c0-4f8f-ac18-02b7a2d2363b None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:56.986195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45396fb-85c0-4f8f-ac18-02b7a2d2363b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:56.986421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45396fb-85c0-4f8f-ac18-02b7a2d2363b None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:56.988073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 88/181] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:56.989073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0283e-2ade-45c2-a5cd-b883812b6917 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:56.989354 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0283e-2ade-45c2-a5cd-b883812b6917 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:56.989626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0283e-2ade-45c2-a5cd-b883812b6917 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:56.995710 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 89/182] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:56.999999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01ef82-1ec8-415a-a670-04482aaa7eb1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:56.999999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01ef82-1ec8-415a-a670-04482aaa7eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:56.999999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01ef82-1ec8-415a-a670-04482aaa7eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:57.019831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-be01ef82-1ec8-415a-a670-04482aaa7eb1 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:57.020959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-be01ef82-1ec8-415a-a670-04482aaa7eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:57.050537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e220e-a4eb-4b25-9bcf-87a720d018c8 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:57.091229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ebd015-7450-40da-9288-3f2a6891352d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:57.092352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebd015-7450-40da-9288-3f2a6891352d None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:57.092842 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebd015-7450-40da-9288-3f2a6891352d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:57.093307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebd015-7450-40da-9288-3f2a6891352d None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:57.094761 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ebd015-7450-40da-9288-3f2a6891352d None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:57.095855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 90/183] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:57.096676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ebd015-7450-40da-9288-3f2a6891352d None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:57.097222 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cece86-cf7a-428e-8d20-342a188cf8b4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:57.097511 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cece86-cf7a-428e-8d20-342a188cf8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:57.097801 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cece86-cf7a-428e-8d20-342a188cf8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:57.099551 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 94/184] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:09:56 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:57.100745 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ebd015-7450-40da-9288-3f2a6891352d None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:57.134521 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a0cece86-cf7a-428e-8d20-342a188cf8b4 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:57.135742 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a0cece86-cf7a-428e-8d20-342a188cf8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:57.153638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:57.258665 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 91/185] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:57.260145 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cece86-cf7a-428e-8d20-342a188cf8b4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:57.291917 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ebd015-7450-40da-9288-3f2a6891352d None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:57.293736 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:57.335761 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e630c19-48b9-49bc-a9bf-0e25148b8438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:57.336436 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e630c19-48b9-49bc-a9bf-0e25148b8438 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:57.336716 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e630c19-48b9-49bc-a9bf-0e25148b8438 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:57.336992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e630c19-48b9-49bc-a9bf-0e25148b8438 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:57.353059 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e630c19-48b9-49bc-a9bf-0e25148b8438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:57.359016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-4e630c19-48b9-49bc-a9bf-0e25148b8438 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:09:57.359016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:57.359016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:57.359016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:57.359016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:57.359016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:57.359016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:57.359016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:57.359016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:57.359016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:57.359016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:57.359016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:57.359016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:57.359016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:57.359016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:57.359016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:57.359967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:57.359967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:57.359967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:57.359967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:57.359967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:57.359967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:57.359967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:57.359967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:57.359967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:57.359967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:57.359967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:57.359967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:57.359967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:57.359967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:57.359967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:57.359967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:57.360880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:57.360880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:57.360880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:57.360880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:57.360880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:57.360880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:57.360880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:57.360880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:57.360880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:57.360880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:57.360880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:09:57.360880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:09:57.360880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 95/186] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:09:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:57.361546 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e630c19-48b9-49bc-a9bf-0e25148b8438 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:57.400020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cece86-cf7a-428e-8d20-342a188cf8b4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:57.442768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5302c3-f869-4e28-a6d7-5e4070c0da97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:57.443657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5302c3-f869-4e28-a6d7-5e4070c0da97 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:57.443931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5302c3-f869-4e28-a6d7-5e4070c0da97 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:57.444200 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5302c3-f869-4e28-a6d7-5e4070c0da97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:57.445001 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5302c3-f869-4e28-a6d7-5e4070c0da97 None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=ab228684d11649dfa97b4acebd727e17, user.id=1b4b6acdd88e421a86924fd2c155794f) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:57.445303 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5302c3-f869-4e28-a6d7-5e4070c0da97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:57.447004 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5302c3-f869-4e28-a6d7-5e4070c0da97 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:57.461820 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 92/187] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:09:57 2022] GET /identity/v3/role_assignments?user.id=1b4b6acdd88e421a86924fd2c155794f&scope.project.id=ab228684d11649dfa97b4acebd727e17&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:57.461820 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5302c3-f869-4e28-a6d7-5e4070c0da97 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:57.477660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:57.487372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e630c19-48b9-49bc-a9bf-0e25148b8438 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:57.530844 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558c587-f614-4c0d-bc5e-df4402650e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:57.531794 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558c587-f614-4c0d-bc5e-df4402650e7d None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:57.532224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558c587-f614-4c0d-bc5e-df4402650e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:57.532677 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558c587-f614-4c0d-bc5e-df4402650e7d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:57.546295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558c587-f614-4c0d-bc5e-df4402650e7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:57.552657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-4558c587-f614-4c0d-bc5e-df4402650e7d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:09:57.552657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:57.552657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:57.552657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:57.552657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:57.552657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:57.552657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:57.552657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:57.552657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:57.552657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:57.552657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:57.552657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:57.552657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:57.552657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:57.552657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:57.552657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:57.553675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:57.553675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:57.553675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:57.553675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:57.553675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:57.553675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:57.553675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:57.553675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:57.553675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:57.553675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:57.553675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:57.553675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:57.553675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:57.553675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:57.553675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:57.553675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:57.554488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:57.554488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:57.554488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:57.554488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:57.554488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:57.554488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:57.554488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:57.554488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:57.554488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:57.554488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:57.554488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:09:57.554488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:09:57.557092 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 96/188] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:09:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:57.558370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558c587-f614-4c0d-bc5e-df4402650e7d None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:57.592171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:57.623506 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5302c3-f869-4e28-a6d7-5e4070c0da97 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:57.681128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de352043-e243-4bf9-9351-ff7d94d4e135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:57.682082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de352043-e243-4bf9-9351-ff7d94d4e135 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:57.682520 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de352043-e243-4bf9-9351-ff7d94d4e135 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:57.682915 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de352043-e243-4bf9-9351-ff7d94d4e135 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:57.684713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de352043-e243-4bf9-9351-ff7d94d4e135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:57.686750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de352043-e243-4bf9-9351-ff7d94d4e135 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:57.689403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 93/189] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:09:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:57.690775 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de352043-e243-4bf9-9351-ff7d94d4e135 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:57.699701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558c587-f614-4c0d-bc5e-df4402650e7d None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:57.741107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc073a8-3db0-4675-9bf8-bdaf1df271db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:57.742129 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc073a8-3db0-4675-9bf8-bdaf1df271db None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:57.742623 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc073a8-3db0-4675-9bf8-bdaf1df271db None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:57.743095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc073a8-3db0-4675-9bf8-bdaf1df271db None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:57.743841 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc073a8-3db0-4675-9bf8-bdaf1df271db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:57.744148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc073a8-3db0-4675-9bf8-bdaf1df271db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:57.745564 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc073a8-3db0-4675-9bf8-bdaf1df271db None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:57.757279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 97/190] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:09:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:57.758599 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc073a8-3db0-4675-9bf8-bdaf1df271db None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:57.788253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:57.810459 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de352043-e243-4bf9-9351-ff7d94d4e135 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:57.847138 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be306d4-ff7d-4d94-b735-18be53a1643d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:57.847992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:57.848541 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be306d4-ff7d-4d94-b735-18be53a1643d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:57.848820 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be306d4-ff7d-4d94-b735-18be53a1643d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:57.849177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be306d4-ff7d-4d94-b735-18be53a1643d None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:57.849981 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be306d4-ff7d-4d94-b735-18be53a1643d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:57.850286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be306d4-ff7d-4d94-b735-18be53a1643d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:57.851677 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be306d4-ff7d-4d94-b735-18be53a1643d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:57.865687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 94/191] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:09:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:57.865687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be306d4-ff7d-4d94-b735-18be53a1643d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:57.876431 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc073a8-3db0-4675-9bf8-bdaf1df271db None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:57.913392 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b6402-fbd1-48f0-8b86-59290af79922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:57.914346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b6402-fbd1-48f0-8b86-59290af79922 None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:57.914452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b6402-fbd1-48f0-8b86-59290af79922 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:57.914682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b6402-fbd1-48f0-8b86-59290af79922 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:57.915370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b6402-fbd1-48f0-8b86-59290af79922 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:57.916820 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b6402-fbd1-48f0-8b86-59290af79922 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:57.941478 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 98/192] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:09:57 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:09:57.950932 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b6402-fbd1-48f0-8b86-59290af79922 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:57.978276 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be306d4-ff7d-4d94-b735-18be53a1643d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:58.023244 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656a862-d394-4256-8dfe-e6c51ed166df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:58.023928 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656a862-d394-4256-8dfe-e6c51ed166df None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:58.024254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656a862-d394-4256-8dfe-e6c51ed166df None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:58.024533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656a862-d394-4256-8dfe-e6c51ed166df None admin] PATH_INFO: `/v3/users/5f54d5261d324cee9ce629a31f49efa6` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:58.046230 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:58.049756 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656a862-d394-4256-8dfe-e6c51ed166df None admin] RBAC: Authorizing `identity:get_user(user_id=5f54d5261d324cee9ce629a31f49efa6)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:58.052268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656a862-d394-4256-8dfe-e6c51ed166df None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:58.059686 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 95/193] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:57 2022] GET /identity/v3/users/5f54d5261d324cee9ce629a31f49efa6 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:58.061186 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656a862-d394-4256-8dfe-e6c51ed166df None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:58.127320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b6402-fbd1-48f0-8b86-59290af79922 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:58.173719 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99c1fc-39e5-4ba0-80ff-8d96929d5a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:58.174757 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99c1fc-39e5-4ba0-80ff-8d96929d5a66 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:58.175381 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99c1fc-39e5-4ba0-80ff-8d96929d5a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:58.175810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99c1fc-39e5-4ba0-80ff-8d96929d5a66 None admin] PATH_INFO: `/v3/users/5f54d5261d324cee9ce629a31f49efa6` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:58.177193 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae99c1fc-39e5-4ba0-80ff-8d96929d5a66 None admin] RBAC: Authorizing `identity:get_user(user_id=5f54d5261d324cee9ce629a31f49efa6)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:58.179030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae99c1fc-39e5-4ba0-80ff-8d96929d5a66 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:58.180868 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 99/194] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:57 2022] GET /identity/v3/users/5f54d5261d324cee9ce629a31f49efa6 => generated 283 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:58.187166 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99c1fc-39e5-4ba0-80ff-8d96929d5a66 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:58.197677 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656a862-d394-4256-8dfe-e6c51ed166df None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:58.201634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:58.245745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615f7fb-5a6a-4320-b72f-aeee91710295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:58.246417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615f7fb-5a6a-4320-b72f-aeee91710295 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:58.246694 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615f7fb-5a6a-4320-b72f-aeee91710295 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:58.246966 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615f7fb-5a6a-4320-b72f-aeee91710295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:58.332456 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99c1fc-39e5-4ba0-80ff-8d96929d5a66 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:58.339388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:58.342665 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615f7fb-5a6a-4320-b72f-aeee91710295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:58.344394 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615f7fb-5a6a-4320-b72f-aeee91710295 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:58.385706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e622dc-5afa-45e5-affa-c39469b5c29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:58.386377 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e622dc-5afa-45e5-affa-c39469b5c29f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:58.386660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e622dc-5afa-45e5-affa-c39469b5c29f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:58.386935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e622dc-5afa-45e5-affa-c39469b5c29f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:58.466463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 96/195] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:09:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:09:58.470298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307dc961-b19a-4b54-bb99-70751d802f3a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:58.470298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307dc961-b19a-4b54-bb99-70751d802f3a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:58.470298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307dc961-b19a-4b54-bb99-70751d802f3a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:58.470298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 97/196] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:09:58.473348 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307dc961-b19a-4b54-bb99-70751d802f3a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:58.486397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e622dc-5afa-45e5-affa-c39469b5c29f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:58.491476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e622dc-5afa-45e5-affa-c39469b5c29f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:58.504978 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:58.601746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:58.606365 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307dc961-b19a-4b54-bb99-70751d802f3a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:58.611280 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 100/197] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:09:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:09:58.617771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438c358-dda0-4afe-80bc-09f9bae16a1e None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:58.618330 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438c358-dda0-4afe-80bc-09f9bae16a1e None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:58.618798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438c358-dda0-4afe-80bc-09f9bae16a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:58.643070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-f438c358-dda0-4afe-80bc-09f9bae16a1e None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:09:58.644680 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f438c358-dda0-4afe-80bc-09f9bae16a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:09:58.646119 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8be5d3-814a-4def-8ada-4d45e0ed1f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:58.646778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8be5d3-814a-4def-8ada-4d45e0ed1f4a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:58.647092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8be5d3-814a-4def-8ada-4d45e0ed1f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:58.647364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8be5d3-814a-4def-8ada-4d45e0ed1f4a None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:58.648575 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8be5d3-814a-4def-8ada-4d45e0ed1f4a None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:58.650104 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8be5d3-814a-4def-8ada-4d45e0ed1f4a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:58.656735 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 98/198] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:09:58 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:58.656735 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8be5d3-814a-4def-8ada-4d45e0ed1f4a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:58.742711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 101/199] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:09:58.743960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438c358-dda0-4afe-80bc-09f9bae16a1e None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:58.786023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8be5d3-814a-4def-8ada-4d45e0ed1f4a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:58.795314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:58.827066 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb250e-1eef-431a-b679-9cb030f80389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:58.828188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb250e-1eef-431a-b679-9cb030f80389 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:58.828646 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb250e-1eef-431a-b679-9cb030f80389 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:58.829094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb250e-1eef-431a-b679-9cb030f80389 None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:58.830612 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb250e-1eef-431a-b679-9cb030f80389 None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:58.832392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb250e-1eef-431a-b679-9cb030f80389 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:58.839430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 99/200] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:09:58 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:58.840578 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb250e-1eef-431a-b679-9cb030f80389 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:58.878732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438c358-dda0-4afe-80bc-09f9bae16a1e None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:58.886441 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:58.926618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e9894-0677-4358-8f42-186ea8d15fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:58.927603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e9894-0677-4358-8f42-186ea8d15fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:58.928069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e9894-0677-4358-8f42-186ea8d15fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:58.928574 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e9894-0677-4358-8f42-186ea8d15fb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:58.929566 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e9894-0677-4358-8f42-186ea8d15fb8 None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=66624eb1d31e484ba73e6737100a3e5c, user.id=5f54d5261d324cee9ce629a31f49efa6) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:58.930083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e9894-0677-4358-8f42-186ea8d15fb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:58.931906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e9894-0677-4358-8f42-186ea8d15fb8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:58.938349 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 102/201] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:09:58 2022] GET /identity/v3/role_assignments?user.id=5f54d5261d324cee9ce629a31f49efa6&scope.project.id=66624eb1d31e484ba73e6737100a3e5c&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:58.939443 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e9894-0677-4358-8f42-186ea8d15fb8 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:58.987924 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb250e-1eef-431a-b679-9cb030f80389 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:59.048515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f84a6f-fded-4458-bc88-e24bdb58b3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:59.049595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f84a6f-fded-4458-bc88-e24bdb58b3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:59.050174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f84a6f-fded-4458-bc88-e24bdb58b3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:59.050889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f84a6f-fded-4458-bc88-e24bdb58b3a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:59.077074 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f84a6f-fded-4458-bc88-e24bdb58b3a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:09:59.082645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-52f84a6f-fded-4458-bc88-e24bdb58b3a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:09:59.082645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:09:59.082645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:09:59.082645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:09:59.082645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:09:59.082645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:09:59.082645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:09:59.082645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:09:59.082645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:09:59.082645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:09:59.082645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:09:59.082645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:09:59.082645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:09:59.082645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:09:59.082645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:09:59.082645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:09:59.083588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:09:59.083588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:09:59.083588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:09:59.083588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:09:59.083588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:09:59.083588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:09:59.083588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:09:59.083588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:09:59.083588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:09:59.083588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:09:59.083588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:09:59.083588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:09:59.083588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:09:59.083588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:09:59.083588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:09:59.083588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:09:59.084371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:09:59.084371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:09:59.084371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:09:59.084371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:09:59.084371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:09:59.084371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:09:59.084371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:09:59.084371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:09:59.084371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:09:59.084371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:09:59.084371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:09:59.084371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:09:59.087122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 100/202] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:09:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:09:59.094693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f84a6f-fded-4458-bc88-e24bdb58b3a2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:59.121522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e9894-0677-4358-8f42-186ea8d15fb8 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:59.123704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:59.196493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d067c-3a6a-4fb1-907d-ba3a2bf937c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:59.197089 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d067c-3a6a-4fb1-907d-ba3a2bf937c7 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:59.197382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d067c-3a6a-4fb1-907d-ba3a2bf937c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:59.197642 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d067c-3a6a-4fb1-907d-ba3a2bf937c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:59.198492 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d067c-3a6a-4fb1-907d-ba3a2bf937c7 None admin] RBAC: Adding query filter params (role.id=7ae6dccec1244987a5bc7bf5c3991703, scope.project.id=66624eb1d31e484ba73e6737100a3e5c, user.id=5f54d5261d324cee9ce629a31f49efa6) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:59.198801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d067c-3a6a-4fb1-907d-ba3a2bf937c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:59.200262 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d067c-3a6a-4fb1-907d-ba3a2bf937c7 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:59.211680 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 103/203] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:09:58 2022] GET /identity/v3/role_assignments?user.id=5f54d5261d324cee9ce629a31f49efa6&scope.project.id=66624eb1d31e484ba73e6737100a3e5c&role.id=7ae6dccec1244987a5bc7bf5c3991703 => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:59.263098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:59.270852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f84a6f-fded-4458-bc88-e24bdb58b3a2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:59.323023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748a1cd-a8fd-475a-8495-24d07a554040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:59.324071 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748a1cd-a8fd-475a-8495-24d07a554040 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:59.324649 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748a1cd-a8fd-475a-8495-24d07a554040 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:59.325111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748a1cd-a8fd-475a-8495-24d07a554040 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:59.326112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3748a1cd-a8fd-475a-8495-24d07a554040 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:09:59.326628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3748a1cd-a8fd-475a-8495-24d07a554040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:59.328287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3748a1cd-a8fd-475a-8495-24d07a554040 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:59.346005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 101/204] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:09:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:59.352561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d067c-3a6a-4fb1-907d-ba3a2bf937c7 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:59.489117 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d067c-3a6a-4fb1-907d-ba3a2bf937c7 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:59.553230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ef648-b754-4cef-9092-4baa436b331d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:59.554282 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ef648-b754-4cef-9092-4baa436b331d None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:59.554723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ef648-b754-4cef-9092-4baa436b331d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:59.555174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ef648-b754-4cef-9092-4baa436b331d None admin] PATH_INFO: `/v3/users/f526a3c1fd564b2d9d3bec4abdac8135` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:59.562969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ef648-b754-4cef-9092-4baa436b331d None admin] RBAC: Authorizing `identity:get_user(user_id=f526a3c1fd564b2d9d3bec4abdac8135)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:59.565004 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ef648-b754-4cef-9092-4baa436b331d None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:09:59.568291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 104/205] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:09:59 2022] GET /identity/v3/users/f526a3c1fd564b2d9d3bec4abdac8135 => generated 289 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:09:59.574210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748a1cd-a8fd-475a-8495-24d07a554040 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:09:59.730452 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748a1cd-a8fd-475a-8495-24d07a554040 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:09:59.788057 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e810ab-7541-4055-9ac4-0671b4b6a070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:09:59.788709 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e810ab-7541-4055-9ac4-0671b4b6a070 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:09:59.788993 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e810ab-7541-4055-9ac4-0671b4b6a070 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:09:59.789271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e810ab-7541-4055-9ac4-0671b4b6a070 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:09:59.824625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:09:59.899274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e810ab-7541-4055-9ac4-0671b4b6a070 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:09:59.901307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e810ab-7541-4055-9ac4-0671b4b6a070 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:00.053926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 102/206] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:09:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:00.059858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ef648-b754-4cef-9092-4baa436b331d None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:00.122222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:00.215171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ef648-b754-4cef-9092-4baa436b331d None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:00.273369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3af67-12f9-40e6-8d2c-79af454ec4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:00.274155 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3af67-12f9-40e6-8d2c-79af454ec4ef None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:00.274439 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3af67-12f9-40e6-8d2c-79af454ec4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:00.274775 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3af67-12f9-40e6-8d2c-79af454ec4ef None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:00.278486 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3af67-12f9-40e6-8d2c-79af454ec4ef None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:00.280051 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3af67-12f9-40e6-8d2c-79af454ec4ef None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:00.282361 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 105/207] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:00 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:00.288463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e810ab-7541-4055-9ac4-0671b4b6a070 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:00.369115 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:00.433708 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e810ab-7541-4055-9ac4-0671b4b6a070 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:00.491668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d36211-5c52-4c90-b703-dcbfd4d69945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:00.492321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d36211-5c52-4c90-b703-dcbfd4d69945 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:00.492598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d36211-5c52-4c90-b703-dcbfd4d69945 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:00.492870 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d36211-5c52-4c90-b703-dcbfd4d69945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:00.493827 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d36211-5c52-4c90-b703-dcbfd4d69945 None admin] RBAC: Adding query filter params (role.id=24102e3afb81458ca8d594a2b25b31b2, scope.project.id=ab228684d11649dfa97b4acebd727e17, user.id=f526a3c1fd564b2d9d3bec4abdac8135) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:00.494178 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d36211-5c52-4c90-b703-dcbfd4d69945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:00.495697 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d36211-5c52-4c90-b703-dcbfd4d69945 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:00.507092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 103/208] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:10:00 2022] GET /identity/v3/role_assignments?user.id=f526a3c1fd564b2d9d3bec4abdac8135&scope.project.id=ab228684d11649dfa97b4acebd727e17&role.id=24102e3afb81458ca8d594a2b25b31b2 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:00.521749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86654542-5e47-4cf5-a38a-2ea8ceb3435a None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:00.522074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86654542-5e47-4cf5-a38a-2ea8ceb3435a None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:00.522338 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86654542-5e47-4cf5-a38a-2ea8ceb3435a None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:00.525276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 106/209] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:00.530058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56250afb-1c96-4664-9178-5d7bf1ce1cd8 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:00.530342 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56250afb-1c96-4664-9178-5d7bf1ce1cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:00.530620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56250afb-1c96-4664-9178-5d7bf1ce1cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:00.550573 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-56250afb-1c96-4664-9178-5d7bf1ce1cd8 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:00.551740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-56250afb-1c96-4664-9178-5d7bf1ce1cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:00.577719 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bd40c-4259-4cdf-b259-6106da120a84 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:00.578392 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bd40c-4259-4cdf-b259-6106da120a84 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:00.578392 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bd40c-4259-4cdf-b259-6106da120a84 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:00.580016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 107/210] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:10:00.589478 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872daef-730d-4b63-b59d-0262a9a9b78b None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:00.589742 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872daef-730d-4b63-b59d-0262a9a9b78b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:00.590103 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872daef-730d-4b63-b59d-0262a9a9b78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:00.618052 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-c872daef-730d-4b63-b59d-0262a9a9b78b None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:00.619682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c872daef-730d-4b63-b59d-0262a9a9b78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:00.686080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 104/211] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:00.710408 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56250afb-1c96-4664-9178-5d7bf1ce1cd8 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:00.800641 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 108/212] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:10:00.825051 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:00.841519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872daef-730d-4b63-b59d-0262a9a9b78b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:00.905031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:00.909485 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56250afb-1c96-4664-9178-5d7bf1ce1cd8 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:00.957881 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77628610-f6ff-489d-b33e-ba6ecba206d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:00.958658 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77628610-f6ff-489d-b33e-ba6ecba206d7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:00.958945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77628610-f6ff-489d-b33e-ba6ecba206d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:00.959220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77628610-f6ff-489d-b33e-ba6ecba206d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:00.974786 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77628610-f6ff-489d-b33e-ba6ecba206d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:00.980859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-77628610-f6ff-489d-b33e-ba6ecba206d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:00.980859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:00.980859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:00.980859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:00.980859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:00.980859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:00.980859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:00.980859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:00.980859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:00.980859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:00.980859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:00.980859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:00.980859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:00.980859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:00.980859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:00.980859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:00.981768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:00.981768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:00.981768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:00.981768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:00.981768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:00.981768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:00.981768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:00.981768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:00.981768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:00.981768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:00.981768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:00.981768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:00.981768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:00.981768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:00.981768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:00.981768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:00.982602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:00.982602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:00.982602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:00.982602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:00.982602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:00.982602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:00.982602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:00.982602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:00.982602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:00.982602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:00.982602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:00.982602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:00.982602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 105/213] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:00.990496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77628610-f6ff-489d-b33e-ba6ecba206d7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:00.999576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872daef-730d-4b63-b59d-0262a9a9b78b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:01.043462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e250967-c4f8-49f8-89ec-a06e8808ff92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:01.044539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e250967-c4f8-49f8-89ec-a06e8808ff92 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:01.045018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e250967-c4f8-49f8-89ec-a06e8808ff92 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:01.045484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e250967-c4f8-49f8-89ec-a06e8808ff92 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:01.047424 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e250967-c4f8-49f8-89ec-a06e8808ff92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:01.049553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e250967-c4f8-49f8-89ec-a06e8808ff92 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:01.052048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 109/214] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:01.057843 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e250967-c4f8-49f8-89ec-a06e8808ff92 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:01.127122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77628610-f6ff-489d-b33e-ba6ecba206d7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:01.154987 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:01.174266 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:01.179276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e250967-c4f8-49f8-89ec-a06e8808ff92 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:01.184592 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e25c5-4fc7-483b-aaac-28fae5fdae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:01.185533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e25c5-4fc7-483b-aaac-28fae5fdae63 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:01.186069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e25c5-4fc7-483b-aaac-28fae5fdae63 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:01.186618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4e25c5-4fc7-483b-aaac-28fae5fdae63 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:01.187921 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e25c5-4fc7-483b-aaac-28fae5fdae63 None admin] RBAC: Adding query filter params (name=member) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:01.188712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e25c5-4fc7-483b-aaac-28fae5fdae63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:01.191583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4e25c5-4fc7-483b-aaac-28fae5fdae63 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:01.206666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 106/215] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:01.215658 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e25c5-4fc7-483b-aaac-28fae5fdae63 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:01.235311 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45df4a5-8253-4a3f-978f-69e3026b9316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:01.236332 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45df4a5-8253-4a3f-978f-69e3026b9316 None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:01.236825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45df4a5-8253-4a3f-978f-69e3026b9316 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:01.237295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45df4a5-8253-4a3f-978f-69e3026b9316 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:01.238637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45df4a5-8253-4a3f-978f-69e3026b9316 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:01.240708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45df4a5-8253-4a3f-978f-69e3026b9316 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:01.282188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 110/216] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:10:01 2022] POST /identity/v3/users => generated 297 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:10:01.382565 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4e25c5-4fc7-483b-aaac-28fae5fdae63 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:01.432639 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75299979-67db-4584-8e45-b018370577af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:01.433674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75299979-67db-4584-8e45-b018370577af None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:01.434286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75299979-67db-4584-8e45-b018370577af None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:01.434863 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75299979-67db-4584-8e45-b018370577af None admin] PATH_INFO: `/v3/users/1fd76f0567f8442baa673ee8d7eb153b` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:01.457957 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75299979-67db-4584-8e45-b018370577af None admin] RBAC: Authorizing `identity:get_user(user_id=1fd76f0567f8442baa673ee8d7eb153b)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:01.459865 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75299979-67db-4584-8e45-b018370577af None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:01.462061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 107/217] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:10:01 2022] GET /identity/v3/users/1fd76f0567f8442baa673ee8d7eb153b => generated 297 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:01.471133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45df4a5-8253-4a3f-978f-69e3026b9316 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:01.615127 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45df4a5-8253-4a3f-978f-69e3026b9316 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:01.674088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468559c-f247-4520-8fcc-bac9ca13afc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:01.674744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468559c-f247-4520-8fcc-bac9ca13afc4 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:01.675067 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468559c-f247-4520-8fcc-bac9ca13afc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:01.675344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468559c-f247-4520-8fcc-bac9ca13afc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:01.707531 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:01.709372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446435d-9a1f-4c94-be4b-e6c2d1a16178 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:01.710229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446435d-9a1f-4c94-be4b-e6c2d1a16178 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:01.710719 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446435d-9a1f-4c94-be4b-e6c2d1a16178 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:01.713319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 108/218] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:01.718546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2784e-4cad-45b9-bfc7-a6fb967c0a69 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:01.719127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2784e-4cad-45b9-bfc7-a6fb967c0a69 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:01.719598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2784e-4cad-45b9-bfc7-a6fb967c0a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:01.747175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d4f2784e-4cad-45b9-bfc7-a6fb967c0a69 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:01.748615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d4f2784e-4cad-45b9-bfc7-a6fb967c0a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:01.768572 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468559c-f247-4520-8fcc-bac9ca13afc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:01.770519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468559c-f247-4520-8fcc-bac9ca13afc4 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:01.789575 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:01.871456 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 109/219] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:01.916091 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2784e-4cad-45b9-bfc7-a6fb967c0a69 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:01.989972 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 111/220] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:01.991590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e98335-fd1b-47e2-9d9d-cc5b95129dfe None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:01.991874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e98335-fd1b-47e2-9d9d-cc5b95129dfe None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:01.992147 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e98335-fd1b-47e2-9d9d-cc5b95129dfe None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:01.994986 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 112/221] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:01.996323 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e98335-fd1b-47e2-9d9d-cc5b95129dfe None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:02.067524 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2784e-4cad-45b9-bfc7-a6fb967c0a69 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:02.074826 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:02.119326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e7afa1-6138-4529-8421-6779fdc324a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:02.120678 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7afa1-6138-4529-8421-6779fdc324a5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:02.121093 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7afa1-6138-4529-8421-6779fdc324a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:02.121553 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e7afa1-6138-4529-8421-6779fdc324a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:02.134242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e7afa1-6138-4529-8421-6779fdc324a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:02.140588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-f9e7afa1-6138-4529-8421-6779fdc324a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:10:02.140588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:02.140588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:02.140588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:02.140588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:02.140588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:02.140588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:02.140588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:02.140588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:02.140588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:02.140588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:02.140588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:02.140588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:02.140588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:02.140588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:02.140588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:02.141700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:02.141700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:02.141700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:02.141700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:02.141700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:02.141700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:02.141700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:02.141700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:02.141700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:02.141700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:02.141700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:02.141700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:02.141700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:02.141700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:02.141700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:02.141700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:02.142653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:02.142653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:02.142653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:02.142653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:02.142653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:02.142653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:02.142653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:02.142653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:02.142653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:02.142653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:02.142653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:10:02.142653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:02.152012 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 110/222] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:10:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:02.153986 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e98335-fd1b-47e2-9d9d-cc5b95129dfe None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:02.154807 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e7b3b-bbec-4557-b894-71da614a9f5f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:02.155435 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e7b3b-bbec-4557-b894-71da614a9f5f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:02.156073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e7b3b-bbec-4557-b894-71da614a9f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:02.188398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a24e7b3b-bbec-4557-b894-71da614a9f5f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:02.190710 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a24e7b3b-bbec-4557-b894-71da614a9f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:02.200735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db05832-8971-474c-b864-c6374ecaf277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:02.201406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db05832-8971-474c-b864-c6374ecaf277 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:02.201687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db05832-8971-474c-b864-c6374ecaf277 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:02.201989 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db05832-8971-474c-b864-c6374ecaf277 None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:02.203207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db05832-8971-474c-b864-c6374ecaf277 None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:02.205427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db05832-8971-474c-b864-c6374ecaf277 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:02.213252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 113/223] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:01 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:02.214881 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db05832-8971-474c-b864-c6374ecaf277 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:02.222584 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:02.283956 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 111/224] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:02.291268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e7b3b-bbec-4557-b894-71da614a9f5f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:02.343587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db05832-8971-474c-b864-c6374ecaf277 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:02.375299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:02.389377 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcdce5-05cc-4f88-b825-2cd903e33bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:02.390360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcdce5-05cc-4f88-b825-2cd903e33bdc None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:02.390786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcdce5-05cc-4f88-b825-2cd903e33bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:02.391206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcdce5-05cc-4f88-b825-2cd903e33bdc None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:02.392090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcdce5-05cc-4f88-b825-2cd903e33bdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:02.392539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcdce5-05cc-4f88-b825-2cd903e33bdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:02.394252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcdce5-05cc-4f88-b825-2cd903e33bdc None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:02.405213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 114/225] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:02.406420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcdce5-05cc-4f88-b825-2cd903e33bdc None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:02.447972 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e7b3b-bbec-4557-b894-71da614a9f5f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:02.484905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277409a-3199-4013-83c6-eb7fddeb63a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:02.485874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277409a-3199-4013-83c6-eb7fddeb63a2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:02.486351 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277409a-3199-4013-83c6-eb7fddeb63a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:02.486788 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277409a-3199-4013-83c6-eb7fddeb63a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:02.487807 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277409a-3199-4013-83c6-eb7fddeb63a2 None admin] RBAC: Adding query filter params (role.id=19db9fb1cb1541ee94673df232bd6952, scope.project.id=66624eb1d31e484ba73e6737100a3e5c, user.id=1fd76f0567f8442baa673ee8d7eb153b) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:02.488264 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277409a-3199-4013-83c6-eb7fddeb63a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:02.489984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277409a-3199-4013-83c6-eb7fddeb63a2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:02.496881 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 112/226] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:10:02 2022] GET /identity/v3/role_assignments?user.id=1fd76f0567f8442baa673ee8d7eb153b&scope.project.id=66624eb1d31e484ba73e6737100a3e5c&role.id=19db9fb1cb1541ee94673df232bd6952 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:02.503055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277409a-3199-4013-83c6-eb7fddeb63a2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:02.532190 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcdce5-05cc-4f88-b825-2cd903e33bdc None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:02.537920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:02.598533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55a1fb-2884-4a37-8f9d-f4db5f4d98b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:02.599214 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55a1fb-2884-4a37-8f9d-f4db5f4d98b5 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:02.599495 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55a1fb-2884-4a37-8f9d-f4db5f4d98b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:02.599767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55a1fb-2884-4a37-8f9d-f4db5f4d98b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:02.622824 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55a1fb-2884-4a37-8f9d-f4db5f4d98b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:02.629625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-4d55a1fb-2884-4a37-8f9d-f4db5f4d98b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:02.629625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:02.629625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:02.629625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:02.629625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:02.629625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:02.629625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:02.629625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:02.629625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:02.629625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:02.629625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:02.629625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:02.629625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:02.629625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:02.629625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:02.629625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:02.630764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:02.630764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:02.630764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:02.630764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:02.630764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:02.630764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:02.630764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:02.630764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:02.630764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:02.630764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:02.630764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:02.630764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:02.630764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:02.630764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:02.630764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:02.630764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:02.631628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:02.631628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:02.631628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:02.631628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:02.631628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:02.631628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:02.631628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:02.631628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:02.631628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:02.631628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:02.631628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:02.631628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:02.633509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 115/227] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:10:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:02.643607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55a1fb-2884-4a37-8f9d-f4db5f4d98b5 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:02.713106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:02.756286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277409a-3199-4013-83c6-eb7fddeb63a2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:02.784284 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55a1fb-2884-4a37-8f9d-f4db5f4d98b5 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:02.821466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d53244-6ed7-46fa-a631-e08c13376b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:02.823824 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d53244-6ed7-46fa-a631-e08c13376b8f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:02.824148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d53244-6ed7-46fa-a631-e08c13376b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:02.824430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d53244-6ed7-46fa-a631-e08c13376b8f None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:02.825221 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d53244-6ed7-46fa-a631-e08c13376b8f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:02.825532 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d53244-6ed7-46fa-a631-e08c13376b8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:02.827037 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d53244-6ed7-46fa-a631-e08c13376b8f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:02.836920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff73323-8d04-4407-8bec-2b603abd3bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:02.837620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff73323-8d04-4407-8bec-2b603abd3bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:02.837966 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff73323-8d04-4407-8bec-2b603abd3bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:02.838301 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff73323-8d04-4407-8bec-2b603abd3bc1 None admin] PATH_INFO: `/v3/users/5f54d5261d324cee9ce629a31f49efa6` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:02.841198 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff73323-8d04-4407-8bec-2b603abd3bc1 None admin] RBAC: Authorizing `identity:get_user(user_id=5f54d5261d324cee9ce629a31f49efa6)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:02.842777 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff73323-8d04-4407-8bec-2b603abd3bc1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:02.844597 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 113/228] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:10:02 2022] GET /identity/v3/users/5f54d5261d324cee9ce629a31f49efa6 => generated 283 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:02.854779 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 116/229] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:02.856429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff73323-8d04-4407-8bec-2b603abd3bc1 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:02.861892 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d53244-6ed7-46fa-a631-e08c13376b8f None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:03.024123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d53244-6ed7-46fa-a631-e08c13376b8f None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:03.042810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:03.045445 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:03.075224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0bfe38-4ca8-48dd-90e0-f6dfc35bf6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:03.075224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bfe38-4ca8-48dd-90e0-f6dfc35bf6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:03.075224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bfe38-4ca8-48dd-90e0-f6dfc35bf6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:03.075224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bfe38-4ca8-48dd-90e0-f6dfc35bf6e3 None admin] PATH_INFO: `/v3/users/5f54d5261d324cee9ce629a31f49efa6` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:03.077219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0bfe38-4ca8-48dd-90e0-f6dfc35bf6e3 None admin] RBAC: Authorizing `identity:get_user(user_id=5f54d5261d324cee9ce629a31f49efa6)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:03.078790 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0bfe38-4ca8-48dd-90e0-f6dfc35bf6e3 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:03.080673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 117/230] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:10:02 2022] GET /identity/v3/users/5f54d5261d324cee9ce629a31f49efa6 => generated 283 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:03.088763 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0bfe38-4ca8-48dd-90e0-f6dfc35bf6e3 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:03.094531 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff73323-8d04-4407-8bec-2b603abd3bc1 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:03.178538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49193dd-a4f3-446f-bf86-e72c88bc0ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:03.178538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49193dd-a4f3-446f-bf86-e72c88bc0ccb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:03.178538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49193dd-a4f3-446f-bf86-e72c88bc0ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:03.179016 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49193dd-a4f3-446f-bf86-e72c88bc0ccb None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:03.180252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49193dd-a4f3-446f-bf86-e72c88bc0ccb None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:03.182793 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49193dd-a4f3-446f-bf86-e72c88bc0ccb None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:03.193862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 114/231] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:02 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:03.201618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49193dd-a4f3-446f-bf86-e72c88bc0ccb None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:03.328338 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0bfe38-4ca8-48dd-90e0-f6dfc35bf6e3 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:03.398894 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49193dd-a4f3-446f-bf86-e72c88bc0ccb None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:03.410211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe79a1e-585a-48ec-bf0a-a3568fb7fdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:03.411154 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe79a1e-585a-48ec-bf0a-a3568fb7fdec None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:03.411569 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe79a1e-585a-48ec-bf0a-a3568fb7fdec None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:03.411978 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe79a1e-585a-48ec-bf0a-a3568fb7fdec None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:03.413354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe79a1e-585a-48ec-bf0a-a3568fb7fdec None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:03.414995 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe79a1e-585a-48ec-bf0a-a3568fb7fdec None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:03.416924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 118/232] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:03 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:03.435412 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe79a1e-585a-48ec-bf0a-a3568fb7fdec None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:03.486660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8940a51-079b-48af-915c-784702b45244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:03.487446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8940a51-079b-48af-915c-784702b45244 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:03.487775 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8940a51-079b-48af-915c-784702b45244 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:03.488759 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8940a51-079b-48af-915c-784702b45244 None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c/users/5f54d5261d324cee9ce629a31f49efa6/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:03.491299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8940a51-079b-48af-915c-784702b45244 None admin] RBAC: Authorizing `identity:create_grant(project_id=66624eb1d31e484ba73e6737100a3e5c, user_id=5f54d5261d324cee9ce629a31f49efa6, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:03.492780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8940a51-079b-48af-915c-784702b45244 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:03.522992 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 115/233] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:10:03 2022] PUT /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c/users/5f54d5261d324cee9ce629a31f49efa6/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:03.535801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:03.625085 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe79a1e-585a-48ec-bf0a-a3568fb7fdec None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:03.676308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12dc6c-d3a1-4fc5-b30c-ea41026696f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:03.677022 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12dc6c-d3a1-4fc5-b30c-ea41026696f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:03.677306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12dc6c-d3a1-4fc5-b30c-ea41026696f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:03.677583 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12dc6c-d3a1-4fc5-b30c-ea41026696f9 None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c/users/5f54d5261d324cee9ce629a31f49efa6/roles/7ae6dccec1244987a5bc7bf5c3991703` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:03.683831 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12dc6c-d3a1-4fc5-b30c-ea41026696f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=66624eb1d31e484ba73e6737100a3e5c, user_id=5f54d5261d324cee9ce629a31f49efa6, role_id=7ae6dccec1244987a5bc7bf5c3991703)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:03.685360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12dc6c-d3a1-4fc5-b30c-ea41026696f9 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:03.706802 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 119/234] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:10:03 2022] PUT /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c/users/5f54d5261d324cee9ce629a31f49efa6/roles/7ae6dccec1244987a5bc7bf5c3991703 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:03.993503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8c1ae-9cf9-4978-a6ea-b6b1a787b6b5 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:03.994060 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8c1ae-9cf9-4978-a6ea-b6b1a787b6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:03.994060 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8c1ae-9cf9-4978-a6ea-b6b1a787b6b5 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:03.999135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 116/235] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:03.999355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395aae68-7f72-4784-a6ef-3b726b8a6027 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:03.999355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395aae68-7f72-4784-a6ef-3b726b8a6027 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:03.999355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395aae68-7f72-4784-a6ef-3b726b8a6027 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:03.999950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 120/236] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:04.006349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b69ea6-f433-4eab-8e3d-0d6fe9d6e427 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:04.006990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b69ea6-f433-4eab-8e3d-0d6fe9d6e427 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:04.007454 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b69ea6-f433-4eab-8e3d-0d6fe9d6e427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:04.011266 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ea44a-2029-4e8f-9cf6-cb0aa235a3c3 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:04.011719 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ea44a-2029-4e8f-9cf6-cb0aa235a3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:04.012215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ea44a-2029-4e8f-9cf6-cb0aa235a3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:04.030351 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f5b69ea6-f433-4eab-8e3d-0d6fe9d6e427 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:04.031939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f5b69ea6-f433-4eab-8e3d-0d6fe9d6e427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:04.043848 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:04.045526 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-333ea44a-2029-4e8f-9cf6-cb0aa235a3c3 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:04.046742 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-333ea44a-2029-4e8f-9cf6-cb0aa235a3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:04.199285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 117/237] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:10:04.216432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 121/238] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:04.223129 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b69ea6-f433-4eab-8e3d-0d6fe9d6e427 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:04.271016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ea44a-2029-4e8f-9cf6-cb0aa235a3c3 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:04.313405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:04.399796 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ea44a-2029-4e8f-9cf6-cb0aa235a3c3 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:04.438844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b69ea6-f433-4eab-8e3d-0d6fe9d6e427 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:04.439267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c798e8-ef57-433c-be1d-177b1d70f67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:04.439936 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c798e8-ef57-433c-be1d-177b1d70f67e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:04.440214 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c798e8-ef57-433c-be1d-177b1d70f67e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:04.440583 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c798e8-ef57-433c-be1d-177b1d70f67e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:04.453571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c798e8-ef57-433c-be1d-177b1d70f67e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:04.459411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-92c798e8-ef57-433c-be1d-177b1d70f67e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:10:04.459411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:04.459411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:04.459411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:04.459411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:04.459411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:04.459411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:04.459411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:04.459411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:04.459411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:04.459411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:04.459411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:04.459411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:04.459411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:04.459411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:04.459411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:04.460366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:04.460366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:04.460366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:04.460366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:04.460366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:04.460366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:04.460366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:04.460366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:04.460366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:04.460366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:04.460366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:04.460366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:04.460366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:04.460366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:04.460366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:04.460366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:04.461229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:04.461229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:04.461229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:04.461229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:04.461229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:04.461229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:04.461229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:04.461229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:04.461229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:04.461229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:04.461229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:10:04.461229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:04.461229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 122/239] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:04.468970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c798e8-ef57-433c-be1d-177b1d70f67e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:04.530650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afae856-8a06-40c2-983c-ae5c7efd8c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:04.531592 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afae856-8a06-40c2-983c-ae5c7efd8c19 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:04.531592 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afae856-8a06-40c2-983c-ae5c7efd8c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:04.532041 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afae856-8a06-40c2-983c-ae5c7efd8c19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:04.533155 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afae856-8a06-40c2-983c-ae5c7efd8c19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:04.534725 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afae856-8a06-40c2-983c-ae5c7efd8c19 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:04.536611 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 118/240] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:04.550861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afae856-8a06-40c2-983c-ae5c7efd8c19 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:04.580258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:04.589202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:04.595953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c798e8-ef57-433c-be1d-177b1d70f67e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:04.633592 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f0ff6-e9d3-43d3-9130-63e0df6550d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:04.634768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f0ff6-e9d3-43d3-9130-63e0df6550d2 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:04.634997 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f0ff6-e9d3-43d3-9130-63e0df6550d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:04.635228 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f0ff6-e9d3-43d3-9130-63e0df6550d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:04.635949 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f0ff6-e9d3-43d3-9130-63e0df6550d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:04.636239 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f0ff6-e9d3-43d3-9130-63e0df6550d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:04.637666 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f0ff6-e9d3-43d3-9130-63e0df6550d2 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:04.647570 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 123/241] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:04.653412 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f0ff6-e9d3-43d3-9130-63e0df6550d2 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:04.765168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f0ff6-e9d3-43d3-9130-63e0df6550d2 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:04.775056 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afae856-8a06-40c2-983c-ae5c7efd8c19 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:04.801619 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108a4c2-f1b2-41a6-9a70-3df03e673dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:04.802322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108a4c2-f1b2-41a6-9a70-3df03e673dee None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:04.802587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108a4c2-f1b2-41a6-9a70-3df03e673dee None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:04.802859 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108a4c2-f1b2-41a6-9a70-3df03e673dee None admin] PATH_INFO: `/v3/users/ac5e66cb9fa14fb69118adf5c6606757` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:04.820088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108a4c2-f1b2-41a6-9a70-3df03e673dee None admin] RBAC: Authorizing `identity:get_user(user_id=ac5e66cb9fa14fb69118adf5c6606757)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:04.821578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108a4c2-f1b2-41a6-9a70-3df03e673dee None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:04.823658 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 124/242] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:10:04 2022] GET /identity/v3/users/ac5e66cb9fa14fb69118adf5c6606757 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:04.829278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108a4c2-f1b2-41a6-9a70-3df03e673dee None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:04.846402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:04.862906 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8e858-399d-4d03-aa43-209d63ea896e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:04.863882 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8e858-399d-4d03-aa43-209d63ea896e None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:04.864287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8e858-399d-4d03-aa43-209d63ea896e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:04.864681 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8e858-399d-4d03-aa43-209d63ea896e None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:04.866010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8e858-399d-4d03-aa43-209d63ea896e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:04.868112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f8e858-399d-4d03-aa43-209d63ea896e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:04.929450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 119/243] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:10:04 2022] POST /identity/v3/users => generated 293 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:10:04.961391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108a4c2-f1b2-41a6-9a70-3df03e673dee None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:05.003743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8fa52-5065-4091-9981-7e28b0e7116d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:05.004429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8fa52-5065-4091-9981-7e28b0e7116d None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:05.004734 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8fa52-5065-4091-9981-7e28b0e7116d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:05.005014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8fa52-5065-4091-9981-7e28b0e7116d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:05.087550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec8fa52-5065-4091-9981-7e28b0e7116d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:05.089435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec8fa52-5065-4091-9981-7e28b0e7116d None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:05.122381 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:05.201635 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 125/244] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:10:05.207602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8e858-399d-4d03-aa43-209d63ea896e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:05.323569 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:05.393946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8e858-399d-4d03-aa43-209d63ea896e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:05.437164 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997b275-bf4f-4dd7-b9d5-3f9e44a4e42f None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:05.437452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997b275-bf4f-4dd7-b9d5-3f9e44a4e42f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:05.437975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997b275-bf4f-4dd7-b9d5-3f9e44a4e42f None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:05.439334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 126/245] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:05.447591 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ae8f4-1e0f-4cfb-aaaa-ddcb8f685760 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:05.448207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ae8f4-1e0f-4cfb-aaaa-ddcb8f685760 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:05.448933 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ae8f4-1e0f-4cfb-aaaa-ddcb8f685760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:05.473274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc384dfb-e7b8-4238-8554-02a83b254a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:05.474358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc384dfb-e7b8-4238-8554-02a83b254a2a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:05.474845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc384dfb-e7b8-4238-8554-02a83b254a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:05.475318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc384dfb-e7b8-4238-8554-02a83b254a2a None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:05.478641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc384dfb-e7b8-4238-8554-02a83b254a2a None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:05.482514 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc384dfb-e7b8-4238-8554-02a83b254a2a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:05.485566 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 120/246] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:05 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:05.500049 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc384dfb-e7b8-4238-8554-02a83b254a2a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:05.503470 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2b8ae8f4-1e0f-4cfb-aaaa-ddcb8f685760 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:05.507987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2b8ae8f4-1e0f-4cfb-aaaa-ddcb8f685760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:05.630132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 127/247] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:05.646496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ae8f4-1e0f-4cfb-aaaa-ddcb8f685760 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:05.657216 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc384dfb-e7b8-4238-8554-02a83b254a2a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:05.710922 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:05.712074 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199c4db-d149-4ef0-b5c2-b286103ce6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:05.713073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199c4db-d149-4ef0-b5c2-b286103ce6ed None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:05.713073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199c4db-d149-4ef0-b5c2-b286103ce6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:05.713241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199c4db-d149-4ef0-b5c2-b286103ce6ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:05.714117 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199c4db-d149-4ef0-b5c2-b286103ce6ed None admin] RBAC: Adding query filter params (role.id=24102e3afb81458ca8d594a2b25b31b2, scope.project.id=66624eb1d31e484ba73e6737100a3e5c, user.id=ac5e66cb9fa14fb69118adf5c6606757) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:05.714438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199c4db-d149-4ef0-b5c2-b286103ce6ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:05.715981 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e199c4db-d149-4ef0-b5c2-b286103ce6ed None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:05.718147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:05.733094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 121/248] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:10:05 2022] GET /identity/v3/role_assignments?user.id=ac5e66cb9fa14fb69118adf5c6606757&scope.project.id=66624eb1d31e484ba73e6737100a3e5c&role.id=24102e3afb81458ca8d594a2b25b31b2 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:05.821367 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ae8f4-1e0f-4cfb-aaaa-ddcb8f685760 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:05.885388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5d658-792b-49ee-83e8-972c47a49028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:05.886578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5d658-792b-49ee-83e8-972c47a49028 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:05.887056 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5d658-792b-49ee-83e8-972c47a49028 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:05.887522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5d658-792b-49ee-83e8-972c47a49028 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:05.907087 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5d658-792b-49ee-83e8-972c47a49028 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:05.916567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-a2d5d658-792b-49ee-83e8-972c47a49028 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:05.916567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:05.916567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:05.916567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:05.916567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:05.916567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:05.916567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:05.916567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:05.916567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:05.916567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:05.916567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:05.916567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:05.916567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:05.916567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:05.916567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:05.916567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:05.917533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:05.917533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:05.917533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:05.917533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:05.917533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:05.917533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:05.917533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:05.917533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:05.917533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:05.917533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:05.917533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:05.917533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:05.917533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:05.917533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:05.917533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:05.917533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:05.918385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:05.918385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:05.918385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:05.918385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:05.918385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:05.918385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:05.918385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:05.918385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:05.918385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:05.918385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:05.918385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:05.918385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:05.930244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 128/249] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:05.931457 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199c4db-d149-4ef0-b5c2-b286103ce6ed None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:06.114810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199c4db-d149-4ef0-b5c2-b286103ce6ed None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:06.165839 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6323b9-b247-488c-ad2a-cd816a56a396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:06.169829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6323b9-b247-488c-ad2a-cd816a56a396 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:06.169829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6323b9-b247-488c-ad2a-cd816a56a396 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:06.169829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6323b9-b247-488c-ad2a-cd816a56a396 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:06.169829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6323b9-b247-488c-ad2a-cd816a56a396 None admin] RBAC: Adding query filter params (name=member) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:06.169829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6323b9-b247-488c-ad2a-cd816a56a396 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:06.176056 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6323b9-b247-488c-ad2a-cd816a56a396 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:06.201909 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 122/250] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:06.210709 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5d658-792b-49ee-83e8-972c47a49028 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:06.429215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5d658-792b-49ee-83e8-972c47a49028 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:06.438255 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:06.499487 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fef56-bf39-4e79-bec7-6fce31239f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:06.500532 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fef56-bf39-4e79-bec7-6fce31239f50 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:06.501073 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fef56-bf39-4e79-bec7-6fce31239f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:06.501539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fef56-bf39-4e79-bec7-6fce31239f50 None admin] PATH_INFO: `/v3/users/1fd76f0567f8442baa673ee8d7eb153b` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:06.503136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fef56-bf39-4e79-bec7-6fce31239f50 None admin] RBAC: Authorizing `identity:get_user(user_id=1fd76f0567f8442baa673ee8d7eb153b)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:06.504981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fef56-bf39-4e79-bec7-6fce31239f50 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:06.506859 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 129/251] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:10:06 2022] GET /identity/v3/users/1fd76f0567f8442baa673ee8d7eb153b => generated 297 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:06.512569 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6323b9-b247-488c-ad2a-cd816a56a396 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:06.600237 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:06.659970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6323b9-b247-488c-ad2a-cd816a56a396 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:06.701045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad2a6c5-3808-4af7-b4c9-d957fc093846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:06.701603 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2a6c5-3808-4af7-b4c9-d957fc093846 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:06.701962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2a6c5-3808-4af7-b4c9-d957fc093846 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:06.702244 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad2a6c5-3808-4af7-b4c9-d957fc093846 None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:06.703615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2a6c5-3808-4af7-b4c9-d957fc093846 None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:06.705181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad2a6c5-3808-4af7-b4c9-d957fc093846 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:06.707624 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 123/252] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:06 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:06.714112 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fef56-bf39-4e79-bec7-6fce31239f50 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:06.898207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fef56-bf39-4e79-bec7-6fce31239f50 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:06.959650 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6189db-e4c8-4731-ae79-e512c510a1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:06.960711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6189db-e4c8-4731-ae79-e512c510a1ef None admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:06.961205 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6189db-e4c8-4731-ae79-e512c510a1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:06.961676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6189db-e4c8-4731-ae79-e512c510a1ef None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c/users/1fd76f0567f8442baa673ee8d7eb153b/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:06.967168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6189db-e4c8-4731-ae79-e512c510a1ef None admin] RBAC: Authorizing `identity:create_grant(project_id=66624eb1d31e484ba73e6737100a3e5c, user_id=1fd76f0567f8442baa673ee8d7eb153b, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:06.969004 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6189db-e4c8-4731-ae79-e512c510a1ef None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:06.991548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 130/253] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:10:06 2022] PUT /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c/users/1fd76f0567f8442baa673ee8d7eb153b/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:07.006902 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c393d-0d4d-42dc-addb-3b74b744f17a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:07.007214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c393d-0d4d-42dc-addb-3b74b744f17a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:07.007455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c393d-0d4d-42dc-addb-3b74b744f17a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:07.009338 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 124/254] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:07.015329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235caa3a-db77-49c5-8b9e-47685f4fd6b1 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:07.016034 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235caa3a-db77-49c5-8b9e-47685f4fd6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:07.016549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235caa3a-db77-49c5-8b9e-47685f4fd6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:07.046458 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-235caa3a-db77-49c5-8b9e-47685f4fd6b1 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:07.048057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-235caa3a-db77-49c5-8b9e-47685f4fd6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:07.093815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:07.157751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 131/255] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:07.185333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12e00e-7dec-4ac5-ae2e-ff98adf0afe6 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:07.185753 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12e00e-7dec-4ac5-ae2e-ff98adf0afe6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:07.185927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12e00e-7dec-4ac5-ae2e-ff98adf0afe6 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:07.187489 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 125/256] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:10:07.195477 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff2ddf-5d20-4f42-b2c8-01720389a868 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:07.195889 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff2ddf-5d20-4f42-b2c8-01720389a868 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:07.196019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff2ddf-5d20-4f42-b2c8-01720389a868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:07.203527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12e00e-7dec-4ac5-ae2e-ff98adf0afe6 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:07.231938 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-efff2ddf-5d20-4f42-b2c8-01720389a868 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:07.233195 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-efff2ddf-5d20-4f42-b2c8-01720389a868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:07.412173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 132/257] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:07.432209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12e00e-7dec-4ac5-ae2e-ff98adf0afe6 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:07.439908 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff2ddf-5d20-4f42-b2c8-01720389a868 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:07.490481 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:07.503927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d60abb-fbcd-41f4-b6a0-f4b6160eef1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:07.504630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d60abb-fbcd-41f4-b6a0-f4b6160eef1d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:07.504917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d60abb-fbcd-41f4-b6a0-f4b6160eef1d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:07.505197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d60abb-fbcd-41f4-b6a0-f4b6160eef1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:07.524242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d60abb-fbcd-41f4-b6a0-f4b6160eef1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:07.538174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e5d60abb-fbcd-41f4-b6a0-f4b6160eef1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:10:07.538174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:07.538174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:07.538174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:07.538174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:07.538174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:07.538174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:07.538174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:07.538174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:07.538174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:07.538174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:07.538174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:07.538174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:07.538174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:07.538174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:07.538174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:07.539163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:07.539163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:07.539163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:07.539163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:07.539163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:07.539163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:07.539163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:07.539163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:07.539163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:07.539163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:07.539163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:07.539163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:07.539163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:07.539163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:07.539163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:07.539163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:07.539975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:07.539975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:07.539975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:07.539975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:07.539975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:07.539975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:07.539975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:07.539975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:07.539975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:07.539975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:07.539975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:10:07.539975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:07.539975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 126/258] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:10:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:07.551223 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d60abb-fbcd-41f4-b6a0-f4b6160eef1d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:07.621325 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:07.652176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff2ddf-5d20-4f42-b2c8-01720389a868 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:07.752583 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f05f06-e3f2-4075-a5d9-3ea7d67dd322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:07.754850 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f05f06-e3f2-4075-a5d9-3ea7d67dd322 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:07.755582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f05f06-e3f2-4075-a5d9-3ea7d67dd322 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:07.756312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f05f06-e3f2-4075-a5d9-3ea7d67dd322 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:07.765745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d60abb-fbcd-41f4-b6a0-f4b6160eef1d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:07.790819 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f05f06-e3f2-4075-a5d9-3ea7d67dd322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:07.796528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-95f05f06-e3f2-4075-a5d9-3ea7d67dd322 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:07.796528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:07.796528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:07.796528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:07.796528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:07.796528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:07.796528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:07.796528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:07.796528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:07.796528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:07.796528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:07.796528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:07.796528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:07.796528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:07.796528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:07.796528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:07.797496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:07.797496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:07.797496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:07.797496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:07.797496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:07.797496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:07.797496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:07.797496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:07.797496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:07.797496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:07.797496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:07.797496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:07.797496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:07.797496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:07.797496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:07.797496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:07.798537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:07.798537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:07.798537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:07.798537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:07.798537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:07.798537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:07.798537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:07.798537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:07.798537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:07.798537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:07.798537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:07.798537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:07.810363 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 133/259] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:10:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:07.817458 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f05f06-e3f2-4075-a5d9-3ea7d67dd322 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:07.878554 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0ce79-72d0-4b97-a7a8-847cfbda7e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:07.879213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0ce79-72d0-4b97-a7a8-847cfbda7e6d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:07.879495 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0ce79-72d0-4b97-a7a8-847cfbda7e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:07.879773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0ce79-72d0-4b97-a7a8-847cfbda7e6d None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:07.880529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0ce79-72d0-4b97-a7a8-847cfbda7e6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:07.880894 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0ce79-72d0-4b97-a7a8-847cfbda7e6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:07.882358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0ce79-72d0-4b97-a7a8-847cfbda7e6d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:07.908354 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 127/260] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:07.916491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0ce79-72d0-4b97-a7a8-847cfbda7e6d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:08.042359 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:08.059842 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f05f06-e3f2-4075-a5d9-3ea7d67dd322 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:08.114146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0ce79-72d0-4b97-a7a8-847cfbda7e6d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:08.175216 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9dfc9b-0b67-4923-8bb0-1af502056839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:08.175907 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9dfc9b-0b67-4923-8bb0-1af502056839 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:08.176227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9dfc9b-0b67-4923-8bb0-1af502056839 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:08.176508 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9dfc9b-0b67-4923-8bb0-1af502056839 None admin] PATH_INFO: `/v3/users/5f54d5261d324cee9ce629a31f49efa6` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:08.182585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9dfc9b-0b67-4923-8bb0-1af502056839 None admin] RBAC: Authorizing `identity:get_user(user_id=5f54d5261d324cee9ce629a31f49efa6)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:08.184088 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9dfc9b-0b67-4923-8bb0-1af502056839 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:08.185829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 128/261] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:10:07 2022] GET /identity/v3/users/5f54d5261d324cee9ce629a31f49efa6 => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:08.191953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aab767-6a32-488f-8a40-748fb20b4553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:08.192613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aab767-6a32-488f-8a40-748fb20b4553 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:08.192893 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aab767-6a32-488f-8a40-748fb20b4553 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:08.193166 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aab767-6a32-488f-8a40-748fb20b4553 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:08.193949 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aab767-6a32-488f-8a40-748fb20b4553 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:08.194299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aab767-6a32-488f-8a40-748fb20b4553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:08.194398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9dfc9b-0b67-4923-8bb0-1af502056839 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:08.195692 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aab767-6a32-488f-8a40-748fb20b4553 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:08.214202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 134/262] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:08.215854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aab767-6a32-488f-8a40-748fb20b4553 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:08.256184 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:08.394420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9dfc9b-0b67-4923-8bb0-1af502056839 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:08.459956 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:08.481661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aab767-6a32-488f-8a40-748fb20b4553 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:08.501966 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cdeef-6da4-465a-b1db-2f77fcbf91e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:08.502724 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cdeef-6da4-465a-b1db-2f77fcbf91e7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:08.503146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cdeef-6da4-465a-b1db-2f77fcbf91e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:08.503693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cdeef-6da4-465a-b1db-2f77fcbf91e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:08.569350 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f656947-50bb-4a3a-8861-0db070035884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:08.570054 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f656947-50bb-4a3a-8861-0db070035884 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:08.570336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f656947-50bb-4a3a-8861-0db070035884 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:08.570604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f656947-50bb-4a3a-8861-0db070035884 None admin] PATH_INFO: `/v3/users/5f54d5261d324cee9ce629a31f49efa6` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:08.572557 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f656947-50bb-4a3a-8861-0db070035884 None admin] RBAC: Authorizing `identity:get_user(user_id=5f54d5261d324cee9ce629a31f49efa6)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:08.574157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f656947-50bb-4a3a-8861-0db070035884 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:08.576868 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 135/263] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:10:08 2022] GET /identity/v3/users/5f54d5261d324cee9ce629a31f49efa6 => generated 283 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:08.586382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f656947-50bb-4a3a-8861-0db070035884 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:08.698331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:08.702750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cdeef-6da4-465a-b1db-2f77fcbf91e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:08.704497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cdeef-6da4-465a-b1db-2f77fcbf91e7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:08.706415 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f656947-50bb-4a3a-8861-0db070035884 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:08.776588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ffc47-e692-4f02-a894-f9bd0363067f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:08.777567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ffc47-e692-4f02-a894-f9bd0363067f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:08.777714 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ffc47-e692-4f02-a894-f9bd0363067f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:08.778023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ffc47-e692-4f02-a894-f9bd0363067f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:08.978541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ffc47-e692-4f02-a894-f9bd0363067f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:08.979979 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ffc47-e692-4f02-a894-f9bd0363067f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:08.992211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 129/264] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 798 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:08.999581 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cdeef-6da4-465a-b1db-2f77fcbf91e7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:09.044203 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:09.146210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:09.154810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cdeef-6da4-465a-b1db-2f77fcbf91e7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:09.163944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 136/265] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:09.174687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7094-204d-47a4-a098-0837fb2b36ff None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:09.174687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7094-204d-47a4-a098-0837fb2b36ff None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:09.174687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a7094-204d-47a4-a098-0837fb2b36ff None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:09.180829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 137/266] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:09 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:09.180829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0976831-252e-4b14-a368-3071ccc649bc None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:09.180829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0976831-252e-4b14-a368-3071ccc649bc None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:09.180829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0976831-252e-4b14-a368-3071ccc649bc None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:09.180829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 138/267] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:09.180829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0976831-252e-4b14-a368-3071ccc649bc None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:09.205216 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10c952-1dcf-4774-8bc9-3b65595748bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:09.205973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10c952-1dcf-4774-8bc9-3b65595748bf None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:09.206356 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10c952-1dcf-4774-8bc9-3b65595748bf None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:09.206635 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10c952-1dcf-4774-8bc9-3b65595748bf None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:09.208073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10c952-1dcf-4774-8bc9-3b65595748bf None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:09.209937 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10c952-1dcf-4774-8bc9-3b65595748bf None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:09.211715 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 130/268] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:08 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:09.213214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cffe1b-2a17-496d-a01e-28cf2acfd46c None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:09.213507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cffe1b-2a17-496d-a01e-28cf2acfd46c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:09.214008 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cffe1b-2a17-496d-a01e-28cf2acfd46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:09.240859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a6cffe1b-2a17-496d-a01e-28cf2acfd46c None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:09.242025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a6cffe1b-2a17-496d-a01e-28cf2acfd46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:09.323928 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0976831-252e-4b14-a368-3071ccc649bc None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:09.338992 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 131/269] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:09.346185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3447312-5943-479d-a1a8-053300d6cf4d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:09.346185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3447312-5943-479d-a1a8-053300d6cf4d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:09.346185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3447312-5943-479d-a1a8-053300d6cf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:09.364903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a3447312-5943-479d-a1a8-053300d6cf4d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:09.366270 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a3447312-5943-479d-a1a8-053300d6cf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:09.384768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:09.394553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676e738-6552-4f71-becf-90adf51859db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:09.395664 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676e738-6552-4f71-becf-90adf51859db None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:09.395826 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676e738-6552-4f71-becf-90adf51859db None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:09.396176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676e738-6552-4f71-becf-90adf51859db None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:09.398220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676e738-6552-4f71-becf-90adf51859db None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:09.401506 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676e738-6552-4f71-becf-90adf51859db None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:09.403378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 139/270] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:09 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:09.404558 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676e738-6552-4f71-becf-90adf51859db None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:09.480678 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 132/271] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:09.481132 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d237e-4bec-472d-a9ec-83a1abefc8e5 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:09.481410 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d237e-4bec-472d-a9ec-83a1abefc8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:09.481683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d237e-4bec-472d-a9ec-83a1abefc8e5 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:09.488201 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 133/272] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:09.488201 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d237e-4bec-472d-a9ec-83a1abefc8e5 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:09.510859 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:09.586263 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676e738-6552-4f71-becf-90adf51859db None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:09.637354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3f25b-09e7-48b9-8df1-c57131c9770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:09.638077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3f25b-09e7-48b9-8df1-c57131c9770c None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:09.638352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3f25b-09e7-48b9-8df1-c57131c9770c None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:09.638450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d237e-4bec-472d-a9ec-83a1abefc8e5 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:09.638625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3f25b-09e7-48b9-8df1-c57131c9770c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:09.639689 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3f25b-09e7-48b9-8df1-c57131c9770c None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=66624eb1d31e484ba73e6737100a3e5c, user.id=5f54d5261d324cee9ce629a31f49efa6) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:09.639994 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3f25b-09e7-48b9-8df1-c57131c9770c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:09.642029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3f25b-09e7-48b9-8df1-c57131c9770c None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:09.650290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 140/273] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:10:09 2022] GET /identity/v3/role_assignments?user.id=5f54d5261d324cee9ce629a31f49efa6&scope.project.id=66624eb1d31e484ba73e6737100a3e5c&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:09.651584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3f25b-09e7-48b9-8df1-c57131c9770c None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:09.697961 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c954b6-8e3e-424f-a282-681ba1dc923b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:09.698667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c954b6-8e3e-424f-a282-681ba1dc923b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:09.698943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c954b6-8e3e-424f-a282-681ba1dc923b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:09.699223 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c954b6-8e3e-424f-a282-681ba1dc923b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:09.713072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c954b6-8e3e-424f-a282-681ba1dc923b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:09.718947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-a0c954b6-8e3e-424f-a282-681ba1dc923b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:09.718947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:09.718947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:09.718947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:09.718947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:09.718947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:09.718947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:09.718947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:09.718947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:09.718947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:09.718947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:09.718947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:09.718947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:09.718947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:09.718947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:09.718947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:09.719942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:09.719942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:09.719942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:09.719942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:09.719942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:09.719942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:09.719942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:09.719942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:09.719942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:09.719942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:09.719942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:09.719942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:09.719942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:09.719942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:09.719942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:09.719942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:09.720806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:09.720806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:09.720806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:09.720806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:09.720806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:09.720806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:09.720806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:09.720806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:09.720806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:09.720806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:09.720806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:09.720806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:09.720806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 134/274] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:09.727217 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571b8a2-0e98-416c-92a2-76761d5a8c5a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:09.727558 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571b8a2-0e98-416c-92a2-76761d5a8c5a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:09.727778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571b8a2-0e98-416c-92a2-76761d5a8c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:09.745659 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:09.752856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7571b8a2-0e98-416c-92a2-76761d5a8c5a None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:09.754779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7571b8a2-0e98-416c-92a2-76761d5a8c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:09.829924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3f25b-09e7-48b9-8df1-c57131c9770c None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:09.851279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 135/275] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:09.852183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571b8a2-0e98-416c-92a2-76761d5a8c5a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:09.854045 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:09.892292 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d874867-670e-438b-85b6-0e21c0a0cf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:09.893253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d874867-670e-438b-85b6-0e21c0a0cf87 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:09.893540 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d874867-670e-438b-85b6-0e21c0a0cf87 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:09.893947 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d874867-670e-438b-85b6-0e21c0a0cf87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:09.894788 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d874867-670e-438b-85b6-0e21c0a0cf87 None admin] RBAC: Adding query filter params (role.id=7ae6dccec1244987a5bc7bf5c3991703, scope.project.id=66624eb1d31e484ba73e6737100a3e5c, user.id=5f54d5261d324cee9ce629a31f49efa6) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:09.895084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d874867-670e-438b-85b6-0e21c0a0cf87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:09.896556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d874867-670e-438b-85b6-0e21c0a0cf87 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:09.905220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 141/276] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:10:09 2022] GET /identity/v3/role_assignments?user.id=5f54d5261d324cee9ce629a31f49efa6&scope.project.id=66624eb1d31e484ba73e6737100a3e5c&role.id=7ae6dccec1244987a5bc7bf5c3991703 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:09.910980 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d874867-670e-438b-85b6-0e21c0a0cf87 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:10.052514 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571b8a2-0e98-416c-92a2-76761d5a8c5a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:10.104340 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d874867-670e-438b-85b6-0e21c0a0cf87 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:10.105918 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79981fb5-bab3-449b-b471-ba0a535985ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:10.106708 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79981fb5-bab3-449b-b471-ba0a535985ad None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:10.107057 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79981fb5-bab3-449b-b471-ba0a535985ad None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:10.107345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79981fb5-bab3-449b-b471-ba0a535985ad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:10.110856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:10.121654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79981fb5-bab3-449b-b471-ba0a535985ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:10.127543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-79981fb5-bab3-449b-b471-ba0a535985ad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:10:10.127543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:10.127543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:10.127543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:10.127543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:10.127543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:10.127543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:10.127543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:10.127543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:10.127543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:10.127543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:10.127543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:10.127543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:10.127543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:10.127543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:10.127543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:10.128650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:10.128650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:10.128650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:10.128650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:10.128650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:10.128650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:10.128650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:10.128650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:10.128650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:10.128650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:10.128650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:10.128650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:10.128650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:10.128650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:10.128650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:10.128650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:10.129434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:10.129434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:10.129434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:10.129434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:10.129434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:10.129434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:10.129434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:10.129434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:10.129434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:10.129434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:10.129434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:10:10.129434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:10.129434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 136/277] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:10.130129 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79981fb5-bab3-449b-b471-ba0a535985ad None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:10.153732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16889d7d-3990-49bd-bc97-929e1e5324c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:10.154626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16889d7d-3990-49bd-bc97-929e1e5324c6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:10.154910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16889d7d-3990-49bd-bc97-929e1e5324c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:10.155193 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16889d7d-3990-49bd-bc97-929e1e5324c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:10.155937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16889d7d-3990-49bd-bc97-929e1e5324c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:10.156244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16889d7d-3990-49bd-bc97-929e1e5324c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:10.157883 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16889d7d-3990-49bd-bc97-929e1e5324c6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:10.169177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 142/278] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:10.170550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7972da-c809-475a-bf35-0bc1044bacbb None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:10.170835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7972da-c809-475a-bf35-0bc1044bacbb None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:10.171277 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7972da-c809-475a-bf35-0bc1044bacbb None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:10.172940 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 143/279] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:10.173575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7972da-c809-475a-bf35-0bc1044bacbb None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:10.229744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:10.260304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79981fb5-bab3-449b-b471-ba0a535985ad None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:10.299781 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7972da-c809-475a-bf35-0bc1044bacbb None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:10.302267 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20d588-0eec-44b7-91be-155df20ca9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:10.303287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20d588-0eec-44b7-91be-155df20ca9cd None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:10.303616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20d588-0eec-44b7-91be-155df20ca9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:10.304003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20d588-0eec-44b7-91be-155df20ca9cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:10.305320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20d588-0eec-44b7-91be-155df20ca9cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:10.306865 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20d588-0eec-44b7-91be-155df20ca9cd None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:10.308485 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 137/280] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:10.309282 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20d588-0eec-44b7-91be-155df20ca9cd None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:10.340548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4490f-bc80-4ee9-9649-3b595f80a765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:10.341556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4490f-bc80-4ee9-9649-3b595f80a765 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:10.341910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4490f-bc80-4ee9-9649-3b595f80a765 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:10.342186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4490f-bc80-4ee9-9649-3b595f80a765 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:10.342924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4490f-bc80-4ee9-9649-3b595f80a765 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:10.343226 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4490f-bc80-4ee9-9649-3b595f80a765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:10.344647 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4490f-bc80-4ee9-9649-3b595f80a765 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:10.354156 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 144/281] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:10.355469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43956263-1ea8-484f-bf24-a4e21907cac2 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:10.355702 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43956263-1ea8-484f-bf24-a4e21907cac2 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:10.355968 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43956263-1ea8-484f-bf24-a4e21907cac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:10.375387 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-43956263-1ea8-484f-bf24-a4e21907cac2 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:10.376610 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-43956263-1ea8-484f-bf24-a4e21907cac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:10.379938 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:10.430317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20d588-0eec-44b7-91be-155df20ca9cd None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:10.453521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 145/282] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:10.454666 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43956263-1ea8-484f-bf24-a4e21907cac2 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:10.471162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b887a9-51fa-4139-ac9c-b09022c0c1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:10.471517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b887a9-51fa-4139-ac9c-b09022c0c1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:10.471860 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b887a9-51fa-4139-ac9c-b09022c0c1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:10.472118 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b887a9-51fa-4139-ac9c-b09022c0c1c2 None admin] PATH_INFO: `/v3/users/2f302746e66a4325bb703f8245fc6f77` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:10.488039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b887a9-51fa-4139-ac9c-b09022c0c1c2 None admin] RBAC: Authorizing `identity:get_user(user_id=2f302746e66a4325bb703f8245fc6f77)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:10.489555 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b887a9-51fa-4139-ac9c-b09022c0c1c2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:10.489717 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:10.491442 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 138/283] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:10:10 2022] GET /identity/v3/users/2f302746e66a4325bb703f8245fc6f77 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:10.492576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b887a9-51fa-4139-ac9c-b09022c0c1c2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:10.565332 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43956263-1ea8-484f-bf24-a4e21907cac2 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:10.600109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f446cf-78c0-4d69-b16e-f2bad224c688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:10.600746 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f446cf-78c0-4d69-b16e-f2bad224c688 None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:10.601032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f446cf-78c0-4d69-b16e-f2bad224c688 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:10.601300 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f446cf-78c0-4d69-b16e-f2bad224c688 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:10.602032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f446cf-78c0-4d69-b16e-f2bad224c688 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:10.603500 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f446cf-78c0-4d69-b16e-f2bad224c688 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:10.623511 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b887a9-51fa-4139-ac9c-b09022c0c1c2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:10.626914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 146/284] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:10:10 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:10:10.628147 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f446cf-78c0-4d69-b16e-f2bad224c688 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:10.654299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:10.668241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe3002-ebba-4fb2-8ece-0c62e1cc9c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:10.668911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe3002-ebba-4fb2-8ece-0c62e1cc9c8b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:10.669190 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe3002-ebba-4fb2-8ece-0c62e1cc9c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:10.669462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe3002-ebba-4fb2-8ece-0c62e1cc9c8b None admin] PATH_INFO: `/v3/users/ac5e66cb9fa14fb69118adf5c6606757` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:10.670743 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe3002-ebba-4fb2-8ece-0c62e1cc9c8b None admin] RBAC: Authorizing `identity:get_user(user_id=ac5e66cb9fa14fb69118adf5c6606757)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:10.672234 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe3002-ebba-4fb2-8ece-0c62e1cc9c8b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:10.678939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 139/285] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:10:10 2022] GET /identity/v3/users/ac5e66cb9fa14fb69118adf5c6606757 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:10.678939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe3002-ebba-4fb2-8ece-0c62e1cc9c8b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:10.763491 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:10.786000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f446cf-78c0-4d69-b16e-f2bad224c688 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:10.830714 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe3002-ebba-4fb2-8ece-0c62e1cc9c8b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:10.834932 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8d435-4e9b-4d34-b922-de4c9c58ba2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:10.835619 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8d435-4e9b-4d34-b922-de4c9c58ba2c None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:10.835969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8d435-4e9b-4d34-b922-de4c9c58ba2c None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:10.836246 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8d435-4e9b-4d34-b922-de4c9c58ba2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:10.848573 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8d435-4e9b-4d34-b922-de4c9c58ba2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:10.855187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-6ed8d435-4e9b-4d34-b922-de4c9c58ba2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:10.855187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:10.855187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:10.855187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:10.855187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:10.855187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:10.855187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:10.855187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:10.855187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:10.855187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:10.855187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:10.855187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:10.855187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:10.855187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:10.855187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:10.855187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:10.856541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:10.856541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:10.856541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:10.856541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:10.856541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:10.856541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:10.856541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:10.856541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:10.856541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:10.856541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:10.856541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:10.856541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:10.856541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:10.856541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:10.856541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:10.856541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:10.857762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:10.857762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:10.857762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:10.857762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:10.857762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:10.857762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:10.857762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:10.857762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:10.857762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:10.857762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:10.857762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:10.857762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:10.857762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 147/286] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:10.857762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8d435-4e9b-4d34-b922-de4c9c58ba2c None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:10.883252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d491ad6-9beb-49a2-b981-cb8ed76c257b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:10.884926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d491ad6-9beb-49a2-b981-cb8ed76c257b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:10.885215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d491ad6-9beb-49a2-b981-cb8ed76c257b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:10.885534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d491ad6-9beb-49a2-b981-cb8ed76c257b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:10.886403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d491ad6-9beb-49a2-b981-cb8ed76c257b None admin] RBAC: Adding query filter params (role.id=19db9fb1cb1541ee94673df232bd6952, user.id=2f302746e66a4325bb703f8245fc6f77, scope.system=all) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:10.886711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d491ad6-9beb-49a2-b981-cb8ed76c257b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:10.888402 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d491ad6-9beb-49a2-b981-cb8ed76c257b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:10.894619 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 140/287] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Aug 1 15:10:10 2022] GET /identity/v3/role_assignments?user.id=2f302746e66a4325bb703f8245fc6f77&scope.system=all&role.id=19db9fb1cb1541ee94673df232bd6952 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:10.895498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d491ad6-9beb-49a2-b981-cb8ed76c257b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:10.990555 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:11.025655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8d435-4e9b-4d34-b922-de4c9c58ba2c None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:11.053859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d491ad6-9beb-49a2-b981-cb8ed76c257b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:11.069745 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1fa50-0dd0-4ee8-ace9-417016fc9394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:11.070465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1fa50-0dd0-4ee8-ace9-417016fc9394 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:11.070745 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1fa50-0dd0-4ee8-ace9-417016fc9394 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:11.071021 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1fa50-0dd0-4ee8-ace9-417016fc9394 None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:11.072975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d1fa50-0dd0-4ee8-ace9-417016fc9394 None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:11.074567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d1fa50-0dd0-4ee8-ace9-417016fc9394 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:11.076668 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 148/288] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:10 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:11.088411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1fa50-0dd0-4ee8-ace9-417016fc9394 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:11.102747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52c5bb-4edd-4a5f-84ef-5830e7bd00c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:11.103403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52c5bb-4edd-4a5f-84ef-5830e7bd00c2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:11.103683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52c5bb-4edd-4a5f-84ef-5830e7bd00c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:11.103964 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52c5bb-4edd-4a5f-84ef-5830e7bd00c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:11.104727 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52c5bb-4edd-4a5f-84ef-5830e7bd00c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:11.105035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52c5bb-4edd-4a5f-84ef-5830e7bd00c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:11.107527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52c5bb-4edd-4a5f-84ef-5830e7bd00c2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:11.118524 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 141/289] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:11.119074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:11.126790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52c5bb-4edd-4a5f-84ef-5830e7bd00c2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:11.257349 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1fa50-0dd0-4ee8-ace9-417016fc9394 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:11.281738 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52c5bb-4edd-4a5f-84ef-5830e7bd00c2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:11.297012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c6e98-fe0b-467b-ab90-dba6fbe51750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:11.298061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c6e98-fe0b-467b-ab90-dba6fbe51750 None admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:11.298522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c6e98-fe0b-467b-ab90-dba6fbe51750 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:11.298978 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c6e98-fe0b-467b-ab90-dba6fbe51750 None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c/users/ac5e66cb9fa14fb69118adf5c6606757/roles/24102e3afb81458ca8d594a2b25b31b2` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:11.301578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c6e98-fe0b-467b-ab90-dba6fbe51750 None admin] RBAC: Authorizing `identity:create_grant(project_id=66624eb1d31e484ba73e6737100a3e5c, user_id=ac5e66cb9fa14fb69118adf5c6606757, role_id=24102e3afb81458ca8d594a2b25b31b2)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:11.303506 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c6e98-fe0b-467b-ab90-dba6fbe51750 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:11.307405 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:11.328330 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 149/290] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:10:11 2022] PUT /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c/users/ac5e66cb9fa14fb69118adf5c6606757/roles/24102e3afb81458ca8d594a2b25b31b2 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:11.333863 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4ecbe-db54-4e2c-802a-5cadbaa56760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:11.335045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4ecbe-db54-4e2c-802a-5cadbaa56760 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:11.335537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4ecbe-db54-4e2c-802a-5cadbaa56760 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:11.335954 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4ecbe-db54-4e2c-802a-5cadbaa56760 None admin] PATH_INFO: `/v3/users/1fd76f0567f8442baa673ee8d7eb153b` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:11.337657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4ecbe-db54-4e2c-802a-5cadbaa56760 None admin] RBAC: Authorizing `identity:get_user(user_id=1fd76f0567f8442baa673ee8d7eb153b)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:11.339583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4ecbe-db54-4e2c-802a-5cadbaa56760 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:11.341627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 142/291] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:10:11 2022] GET /identity/v3/users/1fd76f0567f8442baa673ee8d7eb153b => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:11.348708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c6e98-fe0b-467b-ab90-dba6fbe51750 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:11.464131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:11.500416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c6e98-fe0b-467b-ab90-dba6fbe51750 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:11.550613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2662b-560e-4262-b658-376a4faa4d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:11.551452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2662b-560e-4262-b658-376a4faa4d28 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:11.551732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2662b-560e-4262-b658-376a4faa4d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:11.552008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2662b-560e-4262-b658-376a4faa4d28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:11.653434 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2662b-560e-4262-b658-376a4faa4d28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:11.655001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2662b-560e-4262-b658-376a4faa4d28 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:11.755012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:11.791647 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 150/292] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:11.797600 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4ecbe-db54-4e2c-802a-5cadbaa56760 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:11.936758 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4ecbe-db54-4e2c-802a-5cadbaa56760 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:11.975360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f550761-577b-49b0-960c-c5d3856d750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:11.976028 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f550761-577b-49b0-960c-c5d3856d750d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:11.976304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f550761-577b-49b0-960c-c5d3856d750d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:11.976564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f550761-577b-49b0-960c-c5d3856d750d None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:11.977753 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f550761-577b-49b0-960c-c5d3856d750d None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:11.979244 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f550761-577b-49b0-960c-c5d3856d750d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:11.981243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 143/293] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:11 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:11.987880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2662b-560e-4262-b658-376a4faa4d28 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:12.120993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2662b-560e-4262-b658-376a4faa4d28 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:12.168551 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb3e8f-a94e-4381-b184-419be45b993c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:12.169097 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb3e8f-a94e-4381-b184-419be45b993c None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:12.169372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb3e8f-a94e-4381-b184-419be45b993c None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:12.169644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb3e8f-a94e-4381-b184-419be45b993c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:12.170469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb3e8f-a94e-4381-b184-419be45b993c None admin] RBAC: Adding query filter params (role.id=19db9fb1cb1541ee94673df232bd6952, scope.project.id=66624eb1d31e484ba73e6737100a3e5c, user.id=1fd76f0567f8442baa673ee8d7eb153b) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:12.170776 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb3e8f-a94e-4381-b184-419be45b993c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:12.172215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb3e8f-a94e-4381-b184-419be45b993c None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:12.181489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 151/294] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:10:11 2022] GET /identity/v3/role_assignments?user.id=1fd76f0567f8442baa673ee8d7eb153b&scope.project.id=66624eb1d31e484ba73e6737100a3e5c&role.id=19db9fb1cb1541ee94673df232bd6952 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:13.437823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad10d8-7ca0-49c2-99fc-d311e05fb92b None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:13.438882 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad10d8-7ca0-49c2-99fc-d311e05fb92b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:13.439343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad10d8-7ca0-49c2-99fc-d311e05fb92b None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:13.440877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ada2fb-32db-4087-aff4-2fa097dca5fb None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:13.441355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ada2fb-32db-4087-aff4-2fa097dca5fb None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:13.441434 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ada2fb-32db-4087-aff4-2fa097dca5fb None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:13.442179 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 144/295] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:13.442847 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 152/296] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:10:13.448226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2dd1b-7ad2-4d6c-b0f0-e6af8cfd4df4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:13.448411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d1fb1-23d0-4fff-8862-65732b0f263e None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:13.448493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d1fb1-23d0-4fff-8862-65732b0f263e None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:13.448778 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d1fb1-23d0-4fff-8862-65732b0f263e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:13.449204 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2dd1b-7ad2-4d6c-b0f0-e6af8cfd4df4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:13.449676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2dd1b-7ad2-4d6c-b0f0-e6af8cfd4df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:13.470487 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-554d1fb1-23d0-4fff-8862-65732b0f263e None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:13.470817 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-554d1fb1-23d0-4fff-8862-65732b0f263e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:13.476769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d3f2dd1b-7ad2-4d6c-b0f0-e6af8cfd4df4 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:13.478600 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d3f2dd1b-7ad2-4d6c-b0f0-e6af8cfd4df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:13.508622 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:13.550122 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 153/297] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:10:13.566691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d1fb1-23d0-4fff-8862-65732b0f263e None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:13.578211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:13.608996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 145/298] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:10:13.639389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2dd1b-7ad2-4d6c-b0f0-e6af8cfd4df4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:13.688890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d1fb1-23d0-4fff-8862-65732b0f263e None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:13.726922 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1eb73-8e95-484d-966d-0b27e29a16e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:13.727588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1eb73-8e95-484d-966d-0b27e29a16e8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:13.727868 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1eb73-8e95-484d-966d-0b27e29a16e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:13.728146 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1eb73-8e95-484d-966d-0b27e29a16e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:13.729312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1eb73-8e95-484d-966d-0b27e29a16e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:13.730823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1eb73-8e95-484d-966d-0b27e29a16e8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:13.732448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 154/299] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:13.737466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587cab3-076d-4a3d-9863-0925e583392e None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:13.737466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587cab3-076d-4a3d-9863-0925e583392e None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:13.737466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587cab3-076d-4a3d-9863-0925e583392e None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:13.737466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 155/300] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:13.740638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587cab3-076d-4a3d-9863-0925e583392e None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:13.767309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2dd1b-7ad2-4d6c-b0f0-e6af8cfd4df4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:13.790110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:13.806211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790dad9-14e1-4a5c-9fa7-3a1edbb60cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:13.806874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790dad9-14e1-4a5c-9fa7-3a1edbb60cff None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:13.807155 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790dad9-14e1-4a5c-9fa7-3a1edbb60cff None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:13.807430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790dad9-14e1-4a5c-9fa7-3a1edbb60cff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:13.820142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790dad9-14e1-4a5c-9fa7-3a1edbb60cff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:13.825390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-8790dad9-14e1-4a5c-9fa7-3a1edbb60cff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:13.825390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:13.825390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:13.825390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:13.825390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:13.825390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:13.825390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:13.825390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:13.825390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:13.825390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:13.825390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:13.825390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:13.825390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:13.825390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:13.825390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:13.825390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:13.826519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:13.826519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:13.826519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:13.826519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:13.826519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:13.826519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:13.826519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:13.826519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:13.826519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:13.826519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:13.826519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:13.826519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:13.826519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:13.826519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:13.826519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:13.826519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:13.827323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:13.827323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:13.827323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:13.827323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:13.827323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:13.827323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:13.827323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:13.827323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:13.827323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:13.827323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:13.827323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:13.827323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:13.827323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 146/301] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:13.828414 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fc324-cde6-4559-a81c-7fe8586a8ddd None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:13.828631 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fc324-cde6-4559-a81c-7fe8586a8ddd None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:13.828946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fc324-cde6-4559-a81c-7fe8586a8ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:13.848853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-9b8fc324-cde6-4559-a81c-7fe8586a8ddd None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:13.850111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9b8fc324-cde6-4559-a81c-7fe8586a8ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:13.856646 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587cab3-076d-4a3d-9863-0925e583392e None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:13.889184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:13.900230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96872808-b206-4327-8b37-bb7bdbf161ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:13.900863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96872808-b206-4327-8b37-bb7bdbf161ac None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:13.901142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96872808-b206-4327-8b37-bb7bdbf161ac None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:13.901413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96872808-b206-4327-8b37-bb7bdbf161ac None admin] PATH_INFO: `/v3/groups` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:13.902128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96872808-b206-4327-8b37-bb7bdbf161ac None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:13.912820 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96872808-b206-4327-8b37-bb7bdbf161ac None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:13.922732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 156/302] 10.222.0.48 () {60 vars in 1134 bytes} [Mon Aug 1 15:10:13 2022] POST /identity/v3/groups => generated 229 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:10:13.924299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54f98f-1337-4f83-9ae8-cc0ade447c6b None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:13.924604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54f98f-1337-4f83-9ae8-cc0ade447c6b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:13.924879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54f98f-1337-4f83-9ae8-cc0ade447c6b None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:13.929920 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 157/303] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:13.929920 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54f98f-1337-4f83-9ae8-cc0ade447c6b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:13.932615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 147/304] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:13.939576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b99910-f226-4c60-bfd3-06c961da9253 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:13.939576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b99910-f226-4c60-bfd3-06c961da9253 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:13.939576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b99910-f226-4c60-bfd3-06c961da9253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:13.966547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c5b99910-f226-4c60-bfd3-06c961da9253 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:13.967762 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c5b99910-f226-4c60-bfd3-06c961da9253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:14.059308 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 148/305] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:14.063950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:14.071127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b99910-f226-4c60-bfd3-06c961da9253 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:14.072871 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54f98f-1337-4f83-9ae8-cc0ade447c6b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:14.130077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfe396-c25d-4de8-9f1b-b380a4bd6a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:14.130724 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfe396-c25d-4de8-9f1b-b380a4bd6a65 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:14.131002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfe396-c25d-4de8-9f1b-b380a4bd6a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:14.131966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfe396-c25d-4de8-9f1b-b380a4bd6a65 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:14.132065 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfe396-c25d-4de8-9f1b-b380a4bd6a65 None admin] RBAC: Adding query filter params (name=member) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:14.132433 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfe396-c25d-4de8-9f1b-b380a4bd6a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:14.134373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfe396-c25d-4de8-9f1b-b380a4bd6a65 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:14.144769 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 158/306] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:14.145716 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfe396-c25d-4de8-9f1b-b380a4bd6a65 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:14.207607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b99910-f226-4c60-bfd3-06c961da9253 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:14.209238 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:14.258258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3343ee2-d718-48b6-ad1a-19849df419d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:14.259054 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3343ee2-d718-48b6-ad1a-19849df419d1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:14.259457 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3343ee2-d718-48b6-ad1a-19849df419d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:14.259813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3343ee2-d718-48b6-ad1a-19849df419d1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:14.277544 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3343ee2-d718-48b6-ad1a-19849df419d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:14.282876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-b3343ee2-d718-48b6-ad1a-19849df419d1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:10:14.282876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:14.282876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:14.282876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:14.282876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:14.282876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:14.282876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:14.282876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:14.282876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:14.282876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:14.282876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:14.282876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:14.282876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:14.282876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:14.282876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:14.282876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:14.284699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:14.284699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:14.284699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:14.284699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:14.284699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:14.284699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:14.284699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:14.284699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:14.284699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:14.284699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:14.284699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:14.284699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:14.284699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:14.284699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:14.284699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:14.284699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:14.286983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:14.286983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:14.286983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:14.286983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:14.286983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:14.286983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:14.286983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:14.286983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:14.286983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:14.286983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:14.286983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:10:14.286983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:14.286983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 149/307] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:14.286983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3343ee2-d718-48b6-ad1a-19849df419d1 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:14.302052 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfe396-c25d-4de8-9f1b-b380a4bd6a65 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:14.346601 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb98390-e771-4134-b48d-b4b1540ecc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:14.347264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb98390-e771-4134-b48d-b4b1540ecc0c None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:14.347543 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb98390-e771-4134-b48d-b4b1540ecc0c None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:14.347813 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb98390-e771-4134-b48d-b4b1540ecc0c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:14.359964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb98390-e771-4134-b48d-b4b1540ecc0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:14.365095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-0eb98390-e771-4134-b48d-b4b1540ecc0c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:10:14.365095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:14.365095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:14.365095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:14.365095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:14.365095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:14.365095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:14.365095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:14.365095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:14.365095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:14.365095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:14.365095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:14.365095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:14.365095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:14.365095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:14.365095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:14.366158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:14.366158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:14.366158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:14.366158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:14.366158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:14.366158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:14.366158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:14.366158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:14.366158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:14.366158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:14.366158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:14.366158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:14.366158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:14.366158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:14.366158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:14.366158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:14.371834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:14.371834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:14.371834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:14.371834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:14.371834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:14.371834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:14.371834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:14.371834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:14.371834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:14.371834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:14.371834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:10:14.371834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:14.371834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 159/308] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:14.371834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb98390-e771-4134-b48d-b4b1540ecc0c None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:14.397519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:14.420770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3343ee2-d718-48b6-ad1a-19849df419d1 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:14.459459 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08528a-be4b-4d16-9ec3-9d3c97c46421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:14.460078 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08528a-be4b-4d16-9ec3-9d3c97c46421 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:14.460368 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08528a-be4b-4d16-9ec3-9d3c97c46421 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:14.460637 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08528a-be4b-4d16-9ec3-9d3c97c46421 None admin] PATH_INFO: `/v3/users/2f302746e66a4325bb703f8245fc6f77` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:14.461872 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08528a-be4b-4d16-9ec3-9d3c97c46421 None admin] RBAC: Authorizing `identity:get_user(user_id=2f302746e66a4325bb703f8245fc6f77)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:14.463335 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08528a-be4b-4d16-9ec3-9d3c97c46421 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:14.469826 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 150/309] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:10:14 2022] GET /identity/v3/users/2f302746e66a4325bb703f8245fc6f77 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:14.469826 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08528a-be4b-4d16-9ec3-9d3c97c46421 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:14.505050 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:14.509372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb98390-e771-4134-b48d-b4b1540ecc0c None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:14.548437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cecdf0-396f-492c-9cac-22a0671d50a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:14.549105 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cecdf0-396f-492c-9cac-22a0671d50a1 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:14.549412 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cecdf0-396f-492c-9cac-22a0671d50a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:14.549849 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cecdf0-396f-492c-9cac-22a0671d50a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:14.550879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cecdf0-396f-492c-9cac-22a0671d50a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:14.551191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cecdf0-396f-492c-9cac-22a0671d50a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:14.552837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cecdf0-396f-492c-9cac-22a0671d50a1 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:14.571495 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 160/310] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:14.571495 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cecdf0-396f-492c-9cac-22a0671d50a1 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:14.589662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08528a-be4b-4d16-9ec3-9d3c97c46421 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:14.635557 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e2b88-f909-4a25-9b45-42866d7e696e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:14.636193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e2b88-f909-4a25-9b45-42866d7e696e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:14.636479 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e2b88-f909-4a25-9b45-42866d7e696e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:14.636741 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e2b88-f909-4a25-9b45-42866d7e696e None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:14.637477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e2b88-f909-4a25-9b45-42866d7e696e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:14.637794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e2b88-f909-4a25-9b45-42866d7e696e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:14.639205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e2b88-f909-4a25-9b45-42866d7e696e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:14.649130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 151/311] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:14.650015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e2b88-f909-4a25-9b45-42866d7e696e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:14.681476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:14.699168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cecdf0-396f-492c-9cac-22a0671d50a1 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:14.736396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db031b-75dd-4bd9-a341-ea81c3a122c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:14.737206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db031b-75dd-4bd9-a341-ea81c3a122c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:14.737573 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db031b-75dd-4bd9-a341-ea81c3a122c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:14.737995 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db031b-75dd-4bd9-a341-ea81c3a122c8 None admin] PATH_INFO: `/v3/system/users/2f302746e66a4325bb703f8245fc6f77/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:14.739619 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db031b-75dd-4bd9-a341-ea81c3a122c8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2f302746e66a4325bb703f8245fc6f77, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:14.741108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79db031b-75dd-4bd9-a341-ea81c3a122c8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:14.751802 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 161/312] 10.222.0.48 () {58 vars in 1334 bytes} [Mon Aug 1 15:10:14 2022] PUT /identity/v3/system/users/2f302746e66a4325bb703f8245fc6f77/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:14.752605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db031b-75dd-4bd9-a341-ea81c3a122c8 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:14.776123 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:14.795472 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e2b88-f909-4a25-9b45-42866d7e696e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:14.857323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d24f4b-de29-4fb8-a331-2e8264ca2b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:14.858125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d24f4b-de29-4fb8-a331-2e8264ca2b8e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:14.858936 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d24f4b-de29-4fb8-a331-2e8264ca2b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:14.859312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d24f4b-de29-4fb8-a331-2e8264ca2b8e None admin] PATH_INFO: `/v3/users/3efa7667c36b4e2581c375766f921376` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:14.886795 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d24f4b-de29-4fb8-a331-2e8264ca2b8e None admin] RBAC: Authorizing `identity:get_user(user_id=3efa7667c36b4e2581c375766f921376)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:14.888419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d24f4b-de29-4fb8-a331-2e8264ca2b8e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:14.890367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 152/313] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:10:14 2022] GET /identity/v3/users/3efa7667c36b4e2581c375766f921376 => generated 293 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:14.896773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d24f4b-de29-4fb8-a331-2e8264ca2b8e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:14.902632 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db031b-75dd-4bd9-a341-ea81c3a122c8 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:14.949700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f698ed8-1d91-43eb-8677-df1ad72ec2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:14.950397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f698ed8-1d91-43eb-8677-df1ad72ec2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:14.950677 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f698ed8-1d91-43eb-8677-df1ad72ec2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:14.950951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f698ed8-1d91-43eb-8677-df1ad72ec2b5 None admin] PATH_INFO: `/v3/users/ac5e66cb9fa14fb69118adf5c6606757` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:14.952186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f698ed8-1d91-43eb-8677-df1ad72ec2b5 None admin] RBAC: Authorizing `identity:get_user(user_id=ac5e66cb9fa14fb69118adf5c6606757)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:14.953674 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f698ed8-1d91-43eb-8677-df1ad72ec2b5 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:14.956369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 162/314] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:10:14 2022] GET /identity/v3/users/ac5e66cb9fa14fb69118adf5c6606757 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:14.962603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f698ed8-1d91-43eb-8677-df1ad72ec2b5 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:14.997117 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:15.028222 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d24f4b-de29-4fb8-a331-2e8264ca2b8e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:15.072523 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc68276-1d7e-4a62-8484-59eaade66771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:15.073808 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc68276-1d7e-4a62-8484-59eaade66771 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:15.074331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc68276-1d7e-4a62-8484-59eaade66771 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:15.074889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc68276-1d7e-4a62-8484-59eaade66771 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:15.076255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc68276-1d7e-4a62-8484-59eaade66771 None admin] RBAC: Adding query filter params (role.id=24102e3afb81458ca8d594a2b25b31b2, user.id=3efa7667c36b4e2581c375766f921376, scope.system=all) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:15.077241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc68276-1d7e-4a62-8484-59eaade66771 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:15.079354 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc68276-1d7e-4a62-8484-59eaade66771 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:15.086505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 153/315] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Aug 1 15:10:14 2022] GET /identity/v3/role_assignments?user.id=3efa7667c36b4e2581c375766f921376&scope.system=all&role.id=24102e3afb81458ca8d594a2b25b31b2 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:15.149910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f698ed8-1d91-43eb-8677-df1ad72ec2b5 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:15.195044 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dddd1d-695d-452d-8f30-c9c29f8bcf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:15.196065 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dddd1d-695d-452d-8f30-c9c29f8bcf84 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:15.196596 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dddd1d-695d-452d-8f30-c9c29f8bcf84 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:15.197072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dddd1d-695d-452d-8f30-c9c29f8bcf84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:15.280529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dddd1d-695d-452d-8f30-c9c29f8bcf84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:15.282508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dddd1d-695d-452d-8f30-c9c29f8bcf84 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:15.300209 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:15.395912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 163/316] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:15.401894 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc68276-1d7e-4a62-8484-59eaade66771 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:15.421572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:15.522729 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc68276-1d7e-4a62-8484-59eaade66771 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:15.564768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f9c580-ecef-44dd-b4f3-7f900f86b36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:15.565770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9c580-ecef-44dd-b4f3-7f900f86b36d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:15.566321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9c580-ecef-44dd-b4f3-7f900f86b36d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:15.566780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9c580-ecef-44dd-b4f3-7f900f86b36d None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:15.568182 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f9c580-ecef-44dd-b4f3-7f900f86b36d None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:15.569891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f9c580-ecef-44dd-b4f3-7f900f86b36d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:15.572090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 154/317] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:15 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:15.578051 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dddd1d-695d-452d-8f30-c9c29f8bcf84 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:15.688314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dddd1d-695d-452d-8f30-c9c29f8bcf84 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:15.704495 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:15.730060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb37725-e770-4499-b444-597439d3cbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:15.731167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb37725-e770-4499-b444-597439d3cbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:15.731834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb37725-e770-4499-b444-597439d3cbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:15.732374 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb37725-e770-4499-b444-597439d3cbd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:15.733809 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb37725-e770-4499-b444-597439d3cbd8 None admin] RBAC: Adding query filter params (role.id=24102e3afb81458ca8d594a2b25b31b2, scope.project.id=66624eb1d31e484ba73e6737100a3e5c, user.id=ac5e66cb9fa14fb69118adf5c6606757) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:15.734348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb37725-e770-4499-b444-597439d3cbd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:15.736024 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb37725-e770-4499-b444-597439d3cbd8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:15.743155 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 164/318] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:10:15 2022] GET /identity/v3/role_assignments?user.id=ac5e66cb9fa14fb69118adf5c6606757&scope.project.id=66624eb1d31e484ba73e6737100a3e5c&role.id=24102e3afb81458ca8d594a2b25b31b2 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:16.167039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6272f5-59bd-43c9-9a50-42b2cb780717 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:16.167245 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6272f5-59bd-43c9-9a50-42b2cb780717 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:16.167538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6272f5-59bd-43c9-9a50-42b2cb780717 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:16.168840 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 155/319] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:16.174641 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43a6a2-3903-4d5b-aa26-cad8dd84fc31 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:16.174935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43a6a2-3903-4d5b-aa26-cad8dd84fc31 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:16.175208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43a6a2-3903-4d5b-aa26-cad8dd84fc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:16.193680 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-dc43a6a2-3903-4d5b-aa26-cad8dd84fc31 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:16.194895 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-dc43a6a2-3903-4d5b-aa26-cad8dd84fc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:16.269021 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 165/320] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:16.292093 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6272f5-59bd-43c9-9a50-42b2cb780717 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:16.382853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:16.403928 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6272f5-59bd-43c9-9a50-42b2cb780717 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:16.440083 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9595e5-426e-4706-9de5-58ad0dbe8b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:16.440698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9595e5-426e-4706-9de5-58ad0dbe8b4c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:16.440980 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9595e5-426e-4706-9de5-58ad0dbe8b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:16.441254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9595e5-426e-4706-9de5-58ad0dbe8b4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:16.442491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9595e5-426e-4706-9de5-58ad0dbe8b4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:16.443952 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9595e5-426e-4706-9de5-58ad0dbe8b4c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:16.445722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 156/321] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:16.451394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43a6a2-3903-4d5b-aa26-cad8dd84fc31 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:16.560195 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43a6a2-3903-4d5b-aa26-cad8dd84fc31 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:16.569983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:16.596706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd19297-4a3b-4d6a-99bc-e5d366ee589b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:16.597334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd19297-4a3b-4d6a-99bc-e5d366ee589b None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:16.597601 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd19297-4a3b-4d6a-99bc-e5d366ee589b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:16.597905 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd19297-4a3b-4d6a-99bc-e5d366ee589b None admin] PATH_INFO: `/v3/groups` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:16.598617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd19297-4a3b-4d6a-99bc-e5d366ee589b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:16.600067 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd19297-4a3b-4d6a-99bc-e5d366ee589b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:16.618233 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 166/322] 10.222.0.48 () {60 vars in 1134 bytes} [Mon Aug 1 15:10:16 2022] POST /identity/v3/groups => generated 226 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:10:16.901188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62521eda-84eb-4654-ae53-5944928b18e5 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:16.901686 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62521eda-84eb-4654-ae53-5944928b18e5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:16.901819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62521eda-84eb-4654-ae53-5944928b18e5 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:16.903409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 157/323] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:16.909071 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704bb8e-8b26-42e6-9ef7-67d9c6fec3be None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:16.909348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704bb8e-8b26-42e6-9ef7-67d9c6fec3be None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:16.909625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704bb8e-8b26-42e6-9ef7-67d9c6fec3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:16.929374 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-9704bb8e-8b26-42e6-9ef7-67d9c6fec3be None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:16.930593 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9704bb8e-8b26-42e6-9ef7-67d9c6fec3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:17.008821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 167/324] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:17.034907 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62521eda-84eb-4654-ae53-5944928b18e5 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:17.156065 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62521eda-84eb-4654-ae53-5944928b18e5 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:17.193835 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc5332-a99f-443a-a7f8-729fe4b63613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:17.194505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc5332-a99f-443a-a7f8-729fe4b63613 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:17.194779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc5332-a99f-443a-a7f8-729fe4b63613 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:17.195108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc5332-a99f-443a-a7f8-729fe4b63613 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:17.206809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc5332-a99f-443a-a7f8-729fe4b63613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:17.211989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-87dc5332-a99f-443a-a7f8-729fe4b63613 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:17.211989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:17.211989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:17.211989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:17.211989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:17.211989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:17.211989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:17.211989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:17.211989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:17.211989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:17.211989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:17.211989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:17.211989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:17.211989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:17.211989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:17.211989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:17.213165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:17.213165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:17.213165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:17.213165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:17.213165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:17.213165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:17.213165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:17.213165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:17.213165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:17.213165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:17.213165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:17.213165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:17.213165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:17.213165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:17.213165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:17.213165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:17.214257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:17.214257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:17.214257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:17.214257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:17.214257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:17.214257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:17.214257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:17.214257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:17.214257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:17.214257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:17.214257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:17.214257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:17.215508 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 158/325] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:17.221809 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704bb8e-8b26-42e6-9ef7-67d9c6fec3be None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:17.246163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cd990-88c0-497f-8290-0c3a6eec2ae3 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:17.246503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cd990-88c0-497f-8290-0c3a6eec2ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:17.246720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cd990-88c0-497f-8290-0c3a6eec2ae3 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:17.248128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 159/326] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:17.258461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adafc43-e50b-481b-b4ff-859279cf6b2d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:17.258892 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adafc43-e50b-481b-b4ff-859279cf6b2d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:17.259231 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adafc43-e50b-481b-b4ff-859279cf6b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:17.286679 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2adafc43-e50b-481b-b4ff-859279cf6b2d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:17.288019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2adafc43-e50b-481b-b4ff-859279cf6b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:17.302166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:17.353810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:17.357434 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704bb8e-8b26-42e6-9ef7-67d9c6fec3be None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:17.373911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 160/327] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:10:17.394631 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adafc43-e50b-481b-b4ff-859279cf6b2d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:17.401478 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe4e74-d959-4bcc-968a-ea245e7636ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:17.402161 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe4e74-d959-4bcc-968a-ea245e7636ae None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:17.402444 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe4e74-d959-4bcc-968a-ea245e7636ae None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:17.402774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe4e74-d959-4bcc-968a-ea245e7636ae None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:17.404312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe4e74-d959-4bcc-968a-ea245e7636ae None admin] RBAC: Adding query filter params (name=member) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:17.404631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe4e74-d959-4bcc-968a-ea245e7636ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:17.407238 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe4e74-d959-4bcc-968a-ea245e7636ae None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:17.416684 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 168/328] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:17.423128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe4e74-d959-4bcc-968a-ea245e7636ae None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:17.521693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adafc43-e50b-481b-b4ff-859279cf6b2d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:17.548720 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe4e74-d959-4bcc-968a-ea245e7636ae None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:17.562589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935ffc9-22da-4717-bae9-1bee5fefb984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:17.563251 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935ffc9-22da-4717-bae9-1bee5fefb984 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:17.563526 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935ffc9-22da-4717-bae9-1bee5fefb984 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:17.563796 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935ffc9-22da-4717-bae9-1bee5fefb984 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:17.573113 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:17.577962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935ffc9-22da-4717-bae9-1bee5fefb984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:17.583893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-c935ffc9-22da-4717-bae9-1bee5fefb984 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:10:17.583893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:17.583893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:17.583893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:17.583893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:17.583893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:17.583893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:17.583893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:17.583893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:17.583893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:17.583893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:17.583893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:17.583893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:17.583893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:17.583893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:17.583893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:17.584804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:17.584804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:17.584804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:17.584804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:17.584804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:17.584804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:17.584804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:17.584804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:17.584804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:17.584804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:17.584804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:17.584804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:17.584804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:17.584804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:17.584804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:17.584804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:17.585595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:17.585595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:17.585595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:17.585595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:17.585595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:17.585595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:17.585595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:17.585595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:17.585595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:17.585595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:17.585595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:10:17.585595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:17.586229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0633f47-38a5-4054-b78b-82df3cc8cade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:17.586818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0633f47-38a5-4054-b78b-82df3cc8cade None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:17.587110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0633f47-38a5-4054-b78b-82df3cc8cade None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:17.587389 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0633f47-38a5-4054-b78b-82df3cc8cade None admin] PATH_INFO: `/v3/users/2f302746e66a4325bb703f8245fc6f77` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:17.588561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0633f47-38a5-4054-b78b-82df3cc8cade None admin] RBAC: Authorizing `identity:get_user(user_id=2f302746e66a4325bb703f8245fc6f77)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:17.592278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0633f47-38a5-4054-b78b-82df3cc8cade None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:17.592278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 169/330] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:10:17 2022] GET /identity/v3/users/2f302746e66a4325bb703f8245fc6f77 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:17.592554 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 161/330] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:17.598168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0633f47-38a5-4054-b78b-82df3cc8cade None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:17.599252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935ffc9-22da-4717-bae9-1bee5fefb984 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:17.634197 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:17.722946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0633f47-38a5-4054-b78b-82df3cc8cade None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:17.737550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935ffc9-22da-4717-bae9-1bee5fefb984 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:17.779833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9417fc4-a653-4347-b975-621ec9272674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:17.780781 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9417fc4-a653-4347-b975-621ec9272674 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:17.781060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9417fc4-a653-4347-b975-621ec9272674 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:17.781391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9417fc4-a653-4347-b975-621ec9272674 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:17.782123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9417fc4-a653-4347-b975-621ec9272674 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:17.782657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9417fc4-a653-4347-b975-621ec9272674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:17.784229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9417fc4-a653-4347-b975-621ec9272674 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:17.786997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a5ad9-6e2a-402d-9c59-c6fec8f311a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:17.787933 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a5ad9-6e2a-402d-9c59-c6fec8f311a3 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:17.788416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a5ad9-6e2a-402d-9c59-c6fec8f311a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:17.789100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a5ad9-6e2a-402d-9c59-c6fec8f311a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:17.790367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a5ad9-6e2a-402d-9c59-c6fec8f311a3 None admin] RBAC: Adding query filter params (role.id=19db9fb1cb1541ee94673df232bd6952, user.id=2f302746e66a4325bb703f8245fc6f77, scope.system=all) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:17.790962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a5ad9-6e2a-402d-9c59-c6fec8f311a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:17.792786 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a5ad9-6e2a-402d-9c59-c6fec8f311a3 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:17.797928 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 170/331] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:17.802242 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9417fc4-a653-4347-b975-621ec9272674 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:17.814847 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 162/332] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Aug 1 15:10:17 2022] GET /identity/v3/role_assignments?user.id=2f302746e66a4325bb703f8245fc6f77&scope.system=all&role.id=19db9fb1cb1541ee94673df232bd6952 => generated 510 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:17.920628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:17.932569 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9417fc4-a653-4347-b975-621ec9272674 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:17.990203 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337838ba-22b7-46b8-bea2-5d34cf75d22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:17.990800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337838ba-22b7-46b8-bea2-5d34cf75d22f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:17.991133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337838ba-22b7-46b8-bea2-5d34cf75d22f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:17.991413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337838ba-22b7-46b8-bea2-5d34cf75d22f None admin] PATH_INFO: `/v3/users/3efa7667c36b4e2581c375766f921376` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:17.992745 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337838ba-22b7-46b8-bea2-5d34cf75d22f None admin] RBAC: Authorizing `identity:get_user(user_id=3efa7667c36b4e2581c375766f921376)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:17.994324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337838ba-22b7-46b8-bea2-5d34cf75d22f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:17.996263 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 171/333] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:10:17 2022] GET /identity/v3/users/3efa7667c36b4e2581c375766f921376 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:18.002918 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a5ad9-6e2a-402d-9c59-c6fec8f311a3 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:18.095381 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:18.155295 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a5ad9-6e2a-402d-9c59-c6fec8f311a3 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:18.194642 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fc2db-73b1-4726-9490-4502a458f715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:18.195329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fc2db-73b1-4726-9490-4502a458f715 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:18.195606 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fc2db-73b1-4726-9490-4502a458f715 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:18.195875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fc2db-73b1-4726-9490-4502a458f715 None admin] PATH_INFO: `/v3/system/users/3efa7667c36b4e2581c375766f921376/roles/24102e3afb81458ca8d594a2b25b31b2` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:18.197401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fc2db-73b1-4726-9490-4502a458f715 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3efa7667c36b4e2581c375766f921376, role_id=24102e3afb81458ca8d594a2b25b31b2)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:18.198938 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fc2db-73b1-4726-9490-4502a458f715 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:18.218733 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 163/334] 10.222.0.48 () {58 vars in 1334 bytes} [Mon Aug 1 15:10:18 2022] PUT /identity/v3/system/users/3efa7667c36b4e2581c375766f921376/roles/24102e3afb81458ca8d594a2b25b31b2 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:19.454751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c5053-c39c-4d10-af8f-2b3cda47f957 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:19.455897 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c5053-c39c-4d10-af8f-2b3cda47f957 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:19.456391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c5053-c39c-4d10-af8f-2b3cda47f957 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:19.458411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 172/335] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:19.465057 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792ba98-6a14-4a58-84d8-c30f9ec67673 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:19.465629 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792ba98-6a14-4a58-84d8-c30f9ec67673 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:19.465629 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792ba98-6a14-4a58-84d8-c30f9ec67673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:19.494433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6792ba98-6a14-4a58-84d8-c30f9ec67673 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:19.495702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6792ba98-6a14-4a58-84d8-c30f9ec67673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:19.585353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 164/336] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:19.623441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c5053-c39c-4d10-af8f-2b3cda47f957 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:19.776759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c5053-c39c-4d10-af8f-2b3cda47f957 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:19.814148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d1e69-b49f-45ab-94dd-3f17be412e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:19.815301 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d1e69-b49f-45ab-94dd-3f17be412e7e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:19.815789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d1e69-b49f-45ab-94dd-3f17be412e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:19.816273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d1e69-b49f-45ab-94dd-3f17be412e7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:19.833210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d1e69-b49f-45ab-94dd-3f17be412e7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:19.839373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-ed5d1e69-b49f-45ab-94dd-3f17be412e7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:19.839373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:19.839373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:19.839373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:19.839373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:19.839373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:19.839373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:19.839373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:19.839373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:19.839373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:19.839373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:19.839373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:19.839373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:19.839373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:19.839373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:19.839373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:19.840399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:19.840399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:19.840399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:19.840399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:19.840399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:19.840399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:19.840399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:19.840399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:19.840399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:19.840399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:19.840399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:19.840399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:19.840399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:19.840399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:19.840399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:19.840399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:19.841316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:19.841316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:19.841316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:19.841316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:19.841316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:19.841316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:19.841316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:19.841316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:19.841316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:19.841316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:19.841316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:19.841316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:19.849334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 173/337] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:19.850981 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792ba98-6a14-4a58-84d8-c30f9ec67673 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:19.932967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7d540-649b-4d49-963a-8cd7364a6f19 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:19.933622 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7d540-649b-4d49-963a-8cd7364a6f19 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:19.934139 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7d540-649b-4d49-963a-8cd7364a6f19 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:19.937205 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 174/338] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:19.942150 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d3f9f-6613-42e9-b698-2ad47d827089 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:19.942837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d3f9f-6613-42e9-b698-2ad47d827089 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:19.943323 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d3f9f-6613-42e9-b698-2ad47d827089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:19.978184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:19.986744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-f87d3f9f-6613-42e9-b698-2ad47d827089 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:19.987999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f87d3f9f-6613-42e9-b698-2ad47d827089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:20.023950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:20.081770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792ba98-6a14-4a58-84d8-c30f9ec67673 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:20.125096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8dae7d-1e9f-4455-b091-89132886cd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:20.125790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dae7d-1e9f-4455-b091-89132886cd17 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:20.126265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dae7d-1e9f-4455-b091-89132886cd17 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:20.126551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8dae7d-1e9f-4455-b091-89132886cd17 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:20.127358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8dae7d-1e9f-4455-b091-89132886cd17 None admin] RBAC: Adding query filter params (name=member) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:20.127666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8dae7d-1e9f-4455-b091-89132886cd17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:20.131959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8dae7d-1e9f-4455-b091-89132886cd17 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:20.145520 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 165/339] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:20.152279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8dae7d-1e9f-4455-b091-89132886cd17 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:20.240234 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 175/340] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:10:20.277638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8dae7d-1e9f-4455-b091-89132886cd17 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:20.279472 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d3f9f-6613-42e9-b698-2ad47d827089 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:20.297356 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:20.323114 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99ccdc-2cb5-4fc3-8d6e-bb08b4fd17ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:20.323773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99ccdc-2cb5-4fc3-8d6e-bb08b4fd17ca None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:20.324047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99ccdc-2cb5-4fc3-8d6e-bb08b4fd17ca None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:20.324323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99ccdc-2cb5-4fc3-8d6e-bb08b4fd17ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:20.421645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99ccdc-2cb5-4fc3-8d6e-bb08b4fd17ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:20.423198 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99ccdc-2cb5-4fc3-8d6e-bb08b4fd17ca None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:20.447373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d3f9f-6613-42e9-b698-2ad47d827089 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:20.506560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf16b22-cb97-4622-b260-b53726305191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:20.507230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf16b22-cb97-4622-b260-b53726305191 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:20.507511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf16b22-cb97-4622-b260-b53726305191 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:20.507967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf16b22-cb97-4622-b260-b53726305191 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:20.531555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf16b22-cb97-4622-b260-b53726305191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:20.537191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-6cf16b22-cb97-4622-b260-b53726305191 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:20.537191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:20.537191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:20.537191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:20.537191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:20.537191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:20.537191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:20.537191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:20.537191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:20.537191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:20.537191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:20.537191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:20.537191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:20.537191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:20.537191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:20.537191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:20.538411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:20.538411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:20.538411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:20.538411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:20.538411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:20.538411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:20.538411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:20.538411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:20.538411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:20.538411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:20.538411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:20.538411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:20.538411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:20.538411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:20.538411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:20.538411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:20.539405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:20.539405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:20.539405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:20.539405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:20.539405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:20.539405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:20.539405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:20.539405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:20.539405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:20.539405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:20.539405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:20.539405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:20.545910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 176/341] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:10:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:20.548288 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf16b22-cb97-4622-b260-b53726305191 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:20.564021 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 166/342] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:10:20.569962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99ccdc-2cb5-4fc3-8d6e-bb08b4fd17ca None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:20.577807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:20.599944 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:20.712913 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99ccdc-2cb5-4fc3-8d6e-bb08b4fd17ca None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:20.736170 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf16b22-cb97-4622-b260-b53726305191 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:20.789064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7085c-b53c-4c51-9a97-9db429229420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:20.790477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7085c-b53c-4c51-9a97-9db429229420 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:20.790752 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7085c-b53c-4c51-9a97-9db429229420 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:20.791023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7085c-b53c-4c51-9a97-9db429229420 None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:20.795141 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7085c-b53c-4c51-9a97-9db429229420 None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:20.796840 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7085c-b53c-4c51-9a97-9db429229420 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:20.800105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 167/343] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:20 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:20.807703 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cee320-e318-4caa-a585-c4d7011f0d41 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:20.807982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cee320-e318-4caa-a585-c4d7011f0d41 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:20.808253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cee320-e318-4caa-a585-c4d7011f0d41 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:20.811181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 168/344] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:20 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:20.815061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cee320-e318-4caa-a585-c4d7011f0d41 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:20.817415 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d962bef-acdc-4da8-8b4e-257c4bc4ae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:20.818091 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d962bef-acdc-4da8-8b4e-257c4bc4ae72 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:20.818370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d962bef-acdc-4da8-8b4e-257c4bc4ae72 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:20.818633 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d962bef-acdc-4da8-8b4e-257c4bc4ae72 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:20.819393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d962bef-acdc-4da8-8b4e-257c4bc4ae72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:20.819693 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d962bef-acdc-4da8-8b4e-257c4bc4ae72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:20.824414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d962bef-acdc-4da8-8b4e-257c4bc4ae72 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:20.855606 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 177/345] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:20.857280 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb011897-aabd-4163-9811-08a4888ce563 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:20.857561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb011897-aabd-4163-9811-08a4888ce563 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:20.857863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb011897-aabd-4163-9811-08a4888ce563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:20.903095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-bb011897-aabd-4163-9811-08a4888ce563 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:20.904344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-bb011897-aabd-4163-9811-08a4888ce563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:20.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:20.990732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:20.998684 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 178/346] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:20.999707 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb011897-aabd-4163-9811-08a4888ce563 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:21.014061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cee320-e318-4caa-a585-c4d7011f0d41 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:21.070349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8974e96-8c7b-462d-8e00-a62d4651611e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:21.071491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8974e96-8c7b-462d-8e00-a62d4651611e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:21.072130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8974e96-8c7b-462d-8e00-a62d4651611e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:21.072609 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8974e96-8c7b-462d-8e00-a62d4651611e None admin] PATH_INFO: `/v3/groups/eec63ee8bad640a39895e1cda8b0f24b` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:21.081838 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8974e96-8c7b-462d-8e00-a62d4651611e None admin] RBAC: Authorizing `identity:get_group(group_id=eec63ee8bad640a39895e1cda8b0f24b)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:21.084551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8974e96-8c7b-462d-8e00-a62d4651611e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:21.087623 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 169/347] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 15:10:20 2022] GET /identity/v3/groups/eec63ee8bad640a39895e1cda8b0f24b => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:21.089487 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdf047-2b16-4bd3-a602-94bb847f88b9 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:21.090014 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdf047-2b16-4bd3-a602-94bb847f88b9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:21.090637 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdf047-2b16-4bd3-a602-94bb847f88b9 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:21.093662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 170/348] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:21.094866 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdf047-2b16-4bd3-a602-94bb847f88b9 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:21.133847 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb011897-aabd-4163-9811-08a4888ce563 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:21.173119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f915257-c70e-4f9d-aedf-4456b5c53c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:21.174186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f915257-c70e-4f9d-aedf-4456b5c53c1d None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:21.174831 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f915257-c70e-4f9d-aedf-4456b5c53c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:21.174831 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f915257-c70e-4f9d-aedf-4456b5c53c1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:21.226749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdf047-2b16-4bd3-a602-94bb847f88b9 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:21.257955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f915257-c70e-4f9d-aedf-4456b5c53c1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:21.259609 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f915257-c70e-4f9d-aedf-4456b5c53c1d None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:21.275410 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05799f-e09b-4a82-bf6f-daf6bd6d93df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:21.276547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05799f-e09b-4a82-bf6f-daf6bd6d93df None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:21.277421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05799f-e09b-4a82-bf6f-daf6bd6d93df None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:21.277899 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05799f-e09b-4a82-bf6f-daf6bd6d93df None admin] PATH_INFO: `/v3/roles/member` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:21.278161 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:21.295959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05799f-e09b-4a82-bf6f-daf6bd6d93df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:21.301180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-4e05799f-e09b-4a82-bf6f-daf6bd6d93df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:21.301180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:21.301180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:21.301180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:21.301180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:21.301180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:21.301180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:21.301180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:21.301180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:21.301180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:21.301180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:21.301180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:21.301180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:21.301180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:21.301180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:21.301180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:21.302358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:21.302358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:21.302358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:21.302358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:21.302358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:21.302358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:21.302358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:21.302358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:21.302358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:21.302358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:21.302358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:21.302358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:21.302358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:21.302358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:21.302358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:21.302358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:21.303433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:21.303433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:21.303433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:21.303433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:21.303433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:21.303433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:21.303433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:21.303433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:21.303433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:21.303433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:21.303433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:21.303433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:21.305047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 171/349] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:21.306120 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05799f-e09b-4a82-bf6f-daf6bd6d93df None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:21.318165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:21.398232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 179/350] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:21.400412 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37cf14-925c-4aa8-b326-e1188f461cfa None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:21.400871 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37cf14-925c-4aa8-b326-e1188f461cfa None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:21.401288 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37cf14-925c-4aa8-b326-e1188f461cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:21.431310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-8e37cf14-925c-4aa8-b326-e1188f461cfa None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:21.433038 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8e37cf14-925c-4aa8-b326-e1188f461cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:21.446778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05799f-e09b-4a82-bf6f-daf6bd6d93df None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:21.510030 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5219ecf-91b1-4500-a224-486b65c4b92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:21.510590 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5219ecf-91b1-4500-a224-486b65c4b92c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:21.510866 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5219ecf-91b1-4500-a224-486b65c4b92c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:21.512003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5219ecf-91b1-4500-a224-486b65c4b92c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:21.512003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5219ecf-91b1-4500-a224-486b65c4b92c None admin] RBAC: Adding query filter params (group.id=eec63ee8bad640a39895e1cda8b0f24b, role.id=19db9fb1cb1541ee94673df232bd6952, scope.project.id=ab228684d11649dfa97b4acebd727e17) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:21.512449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5219ecf-91b1-4500-a224-486b65c4b92c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:21.514203 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5219ecf-91b1-4500-a224-486b65c4b92c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:21.524368 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 172/351] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 15:10:21 2022] GET /identity/v3/role_assignments?group.id=eec63ee8bad640a39895e1cda8b0f24b&scope.project.id=ab228684d11649dfa97b4acebd727e17&role.id=19db9fb1cb1541ee94673df232bd6952 => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:21.526308 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5219ecf-91b1-4500-a224-486b65c4b92c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:21.550547 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 180/352] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:21.553526 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba014b-2d28-49e7-855f-60d2c891c696 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:21.554100 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba014b-2d28-49e7-855f-60d2c891c696 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:21.554513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba014b-2d28-49e7-855f-60d2c891c696 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:21.556655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 181/353] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:21.557680 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba014b-2d28-49e7-855f-60d2c891c696 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:21.600142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:21.641260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:21.693887 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5219ecf-91b1-4500-a224-486b65c4b92c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:21.724494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba014b-2d28-49e7-855f-60d2c891c696 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:21.741481 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed109652-fba5-4159-95ad-95841048a092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:21.742548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed109652-fba5-4159-95ad-95841048a092 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:21.742999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed109652-fba5-4159-95ad-95841048a092 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:21.743455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed109652-fba5-4159-95ad-95841048a092 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:21.744710 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed109652-fba5-4159-95ad-95841048a092 None admin] RBAC: Adding query filter params (name=member) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:21.745236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed109652-fba5-4159-95ad-95841048a092 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:21.747226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed109652-fba5-4159-95ad-95841048a092 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:21.759661 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 173/354] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:21.760865 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4e3e9-aad3-4379-90ff-1a12ecfad896 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:21.761101 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4e3e9-aad3-4379-90ff-1a12ecfad896 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:21.761375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4e3e9-aad3-4379-90ff-1a12ecfad896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:21.775281 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8214c61-171a-4c3a-8ada-2696e5fc6ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:21.776352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8214c61-171a-4c3a-8ada-2696e5fc6ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:21.776821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8214c61-171a-4c3a-8ada-2696e5fc6ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:21.778082 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8214c61-171a-4c3a-8ada-2696e5fc6ab8 None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:21.780214 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8214c61-171a-4c3a-8ada-2696e5fc6ab8 None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:21.781831 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8214c61-171a-4c3a-8ada-2696e5fc6ab8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:21.783909 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-1ac4e3e9-aad3-4379-90ff-1a12ecfad896 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:21.784113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 182/355] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:21 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:21.784921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8214c61-171a-4c3a-8ada-2696e5fc6ab8 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:21.785271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1ac4e3e9-aad3-4379-90ff-1a12ecfad896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:21.888805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 174/356] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:21.892411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1eec0-95a1-4796-a34b-8b62aefc01fe None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:21.892411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1eec0-95a1-4796-a34b-8b62aefc01fe None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:21.892411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1eec0-95a1-4796-a34b-8b62aefc01fe None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:21.893108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 175/357] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:21.893869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1eec0-95a1-4796-a34b-8b62aefc01fe None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:21.949593 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:21.955736 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8214c61-171a-4c3a-8ada-2696e5fc6ab8 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:21.978882 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:22.007215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b0c88-9718-405c-a843-eb714d5edd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:22.008300 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b0c88-9718-405c-a843-eb714d5edd04 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:22.008811 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b0c88-9718-405c-a843-eb714d5edd04 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:22.009335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b0c88-9718-405c-a843-eb714d5edd04 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:22.023865 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b0c88-9718-405c-a843-eb714d5edd04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:22.030388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-988b0c88-9718-405c-a843-eb714d5edd04 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:22.030388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:22.030388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:22.030388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:22.030388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:22.030388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:22.030388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:22.030388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:22.030388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:22.030388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:22.030388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:22.030388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:22.030388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:22.030388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:22.030388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:22.030388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:22.031356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:22.031356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:22.031356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:22.031356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:22.031356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:22.031356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:22.031356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:22.031356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:22.031356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:22.031356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:22.031356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:22.031356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:22.031356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:22.031356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:22.031356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:22.031356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:22.032157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:22.032157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:22.032157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:22.032157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:22.032157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:22.032157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:22.032157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:22.032157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:22.032157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:22.032157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:22.032157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:22.032157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:22.035843 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 183/358] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:10:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:22.037516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b0c88-9718-405c-a843-eb714d5edd04 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:22.041009 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1eec0-95a1-4796-a34b-8b62aefc01fe None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:22.088001 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97687a7-2cbf-44ee-b869-d3bc1989bbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:22.088706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97687a7-2cbf-44ee-b869-d3bc1989bbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:22.088984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97687a7-2cbf-44ee-b869-d3bc1989bbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:22.089260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97687a7-2cbf-44ee-b869-d3bc1989bbe9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:22.176379 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97687a7-2cbf-44ee-b869-d3bc1989bbe9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:22.178988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97687a7-2cbf-44ee-b869-d3bc1989bbe9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:22.179178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b0c88-9718-405c-a843-eb714d5edd04 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:22.218529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd829428-a5f4-4945-aea7-55f0dd24d429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:22.219195 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd829428-a5f4-4945-aea7-55f0dd24d429 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:22.219473 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd829428-a5f4-4945-aea7-55f0dd24d429 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:22.219744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd829428-a5f4-4945-aea7-55f0dd24d429 None admin] PATH_INFO: `/v3/groups/eec63ee8bad640a39895e1cda8b0f24b` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:22.221148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd829428-a5f4-4945-aea7-55f0dd24d429 None admin] RBAC: Authorizing `identity:get_group(group_id=eec63ee8bad640a39895e1cda8b0f24b)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:22.222686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd829428-a5f4-4945-aea7-55f0dd24d429 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:22.224496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 184/359] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 15:10:22 2022] GET /identity/v3/groups/eec63ee8bad640a39895e1cda8b0f24b => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:22.225664 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3adf26-33bf-49f5-af0f-3a9ab467199e None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:22.225972 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3adf26-33bf-49f5-af0f-3a9ab467199e None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:22.226242 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3adf26-33bf-49f5-af0f-3a9ab467199e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:22.249025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2d3adf26-33bf-49f5-af0f-3a9ab467199e None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:22.249025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2d3adf26-33bf-49f5-af0f-3a9ab467199e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:22.274151 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:22.277825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:22.340876 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 185/360] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:22.345822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3adf26-33bf-49f5-af0f-3a9ab467199e None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:22.346644 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 176/361] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:22.347892 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97687a7-2cbf-44ee-b869-d3bc1989bbe9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:22.495701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3adf26-33bf-49f5-af0f-3a9ab467199e None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:22.502180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97687a7-2cbf-44ee-b869-d3bc1989bbe9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:22.541169 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b516c0a-d8a7-4901-867e-69513a104556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:22.541874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b516c0a-d8a7-4901-867e-69513a104556 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:22.542157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b516c0a-d8a7-4901-867e-69513a104556 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:22.542437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b516c0a-d8a7-4901-867e-69513a104556 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:22.552735 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c53f69-ea3b-47b3-be1c-bd654935a8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:22.553864 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c53f69-ea3b-47b3-be1c-bd654935a8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:22.554276 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c53f69-ea3b-47b3-be1c-bd654935a8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:22.554688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c53f69-ea3b-47b3-be1c-bd654935a8b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:22.555427 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c53f69-ea3b-47b3-be1c-bd654935a8b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:22.555725 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c53f69-ea3b-47b3-be1c-bd654935a8b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:22.557131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c53f69-ea3b-47b3-be1c-bd654935a8b6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:22.560300 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b516c0a-d8a7-4901-867e-69513a104556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:22.565815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-0b516c0a-d8a7-4901-867e-69513a104556 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:10:22.565815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:22.565815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:22.565815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:22.565815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:22.565815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:22.565815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:22.565815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:22.565815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:22.565815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:22.565815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:22.565815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:22.565815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:22.565815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:22.565815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:22.565815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:22.566798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:22.566798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:22.566798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:22.566798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:22.566798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:22.566798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:22.566798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:22.566798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:22.566798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:22.566798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:22.566798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:22.566798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:22.566798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:22.566798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:22.566798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:22.566798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:22.567701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:22.567701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:22.567701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:22.567701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:22.567701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:22.567701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:22.567701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:22.567701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:22.567701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:22.567701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:22.567701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:10:22.567701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:22.568340 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 186/362] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:22.569275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b516c0a-d8a7-4901-867e-69513a104556 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:22.575574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 177/363] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:22.576260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c53f69-ea3b-47b3-be1c-bd654935a8b6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:22.595102 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:22.614247 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:22.723412 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b516c0a-d8a7-4901-867e-69513a104556 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:22.727359 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c53f69-ea3b-47b3-be1c-bd654935a8b6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:22.775757 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0055f3-35ed-454d-bca6-e4d4a3845a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:22.776396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0055f3-35ed-454d-bca6-e4d4a3845a8f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:22.776672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0055f3-35ed-454d-bca6-e4d4a3845a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:22.776943 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0055f3-35ed-454d-bca6-e4d4a3845a8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:22.777769 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0055f3-35ed-454d-bca6-e4d4a3845a8f None admin] RBAC: Adding query filter params (group.id=eec63ee8bad640a39895e1cda8b0f24b, role.id=7ae6dccec1244987a5bc7bf5c3991703, scope.project.id=ab228684d11649dfa97b4acebd727e17) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:22.777992 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850cc6d-8b3e-4f39-aecf-4ffbdb0b57d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:22.778083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0055f3-35ed-454d-bca6-e4d4a3845a8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:22.779464 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850cc6d-8b3e-4f39-aecf-4ffbdb0b57d9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:22.779464 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850cc6d-8b3e-4f39-aecf-4ffbdb0b57d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:22.779754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0055f3-35ed-454d-bca6-e4d4a3845a8f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:22.780037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850cc6d-8b3e-4f39-aecf-4ffbdb0b57d9 None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:22.781987 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850cc6d-8b3e-4f39-aecf-4ffbdb0b57d9 None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:22.783351 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850cc6d-8b3e-4f39-aecf-4ffbdb0b57d9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:22.785212 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 178/364] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:22 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:22.786076 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850cc6d-8b3e-4f39-aecf-4ffbdb0b57d9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:22.787021 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 187/365] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 15:10:22 2022] GET /identity/v3/role_assignments?group.id=eec63ee8bad640a39895e1cda8b0f24b&scope.project.id=ab228684d11649dfa97b4acebd727e17&role.id=7ae6dccec1244987a5bc7bf5c3991703 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:22.788432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0055f3-35ed-454d-bca6-e4d4a3845a8f None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:22.932275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0055f3-35ed-454d-bca6-e4d4a3845a8f None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:22.939750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:22.950200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:22.956773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850cc6d-8b3e-4f39-aecf-4ffbdb0b57d9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:22.986329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db6a1a-5a90-4fef-843c-faa271e80f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:22.987072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db6a1a-5a90-4fef-843c-faa271e80f81 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:22.987365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db6a1a-5a90-4fef-843c-faa271e80f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:22.987938 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db6a1a-5a90-4fef-843c-faa271e80f81 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:22.989258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db6a1a-5a90-4fef-843c-faa271e80f81 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:22.989579 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db6a1a-5a90-4fef-843c-faa271e80f81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:22.991645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db6a1a-5a90-4fef-843c-faa271e80f81 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:22.999356 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbf05e-1fa2-4e38-97e9-11da1c3bed64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:23.000135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbf05e-1fa2-4e38-97e9-11da1c3bed64 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:23.000474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbf05e-1fa2-4e38-97e9-11da1c3bed64 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:23.000810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfbf05e-1fa2-4e38-97e9-11da1c3bed64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:23.004981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 188/366] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:23.005923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db6a1a-5a90-4fef-843c-faa271e80f81 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:23.023755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfbf05e-1fa2-4e38-97e9-11da1c3bed64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:23.033213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-3cfbf05e-1fa2-4e38-97e9-11da1c3bed64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:10:23.033213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:23.033213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:23.033213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:23.033213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:23.033213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:23.033213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:23.033213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:23.033213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:23.033213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:23.033213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:23.033213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:23.033213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:23.033213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:23.033213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:23.033213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:23.034325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:23.034325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:23.034325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:23.034325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:23.034325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:23.034325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:23.034325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:23.034325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:23.034325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:23.034325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:23.034325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:23.034325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:23.034325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:23.034325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:23.034325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:23.034325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:23.035330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:23.035330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:23.035330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:23.035330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:23.035330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:23.035330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:23.035330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:23.035330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:23.035330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:23.035330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:23.035330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:10:23.035330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:23.036026 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 179/367] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:10:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:23.036531 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbf05e-1fa2-4e38-97e9-11da1c3bed64 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:23.154191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db6a1a-5a90-4fef-843c-faa271e80f81 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:23.175116 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfbf05e-1fa2-4e38-97e9-11da1c3bed64 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:23.194351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3316f-2cdf-4d4d-b2f1-a4062ff737bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:23.195292 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3316f-2cdf-4d4d-b2f1-a4062ff737bd None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:23.195570 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3316f-2cdf-4d4d-b2f1-a4062ff737bd None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:23.195843 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3316f-2cdf-4d4d-b2f1-a4062ff737bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:23.213151 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:23.233822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f18929-a53b-4729-be99-6fe6c46e3cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:23.234913 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f18929-a53b-4729-be99-6fe6c46e3cbe None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:23.235424 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f18929-a53b-4729-be99-6fe6c46e3cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:23.235893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f18929-a53b-4729-be99-6fe6c46e3cbe None admin] PATH_INFO: `/v3/groups/eec63ee8bad640a39895e1cda8b0f24b` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:23.238032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f18929-a53b-4729-be99-6fe6c46e3cbe None admin] RBAC: Authorizing `identity:get_group(group_id=eec63ee8bad640a39895e1cda8b0f24b)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:23.240511 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f18929-a53b-4729-be99-6fe6c46e3cbe None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:23.243459 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 180/368] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 15:10:23 2022] GET /identity/v3/groups/eec63ee8bad640a39895e1cda8b0f24b => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:23.245406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f18929-a53b-4729-be99-6fe6c46e3cbe None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:23.280705 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:23.294349 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f3316f-2cdf-4d4d-b2f1-a4062ff737bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:23.296388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f3316f-2cdf-4d4d-b2f1-a4062ff737bd None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:23.395045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f18929-a53b-4729-be99-6fe6c46e3cbe None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:23.446004 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 189/369] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:23.446817 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3316f-2cdf-4d4d-b2f1-a4062ff737bd None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:23.448256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0401f-765c-4371-b859-fd591f227e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:23.449458 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0401f-765c-4371-b859-fd591f227e5a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:23.451050 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0401f-765c-4371-b859-fd591f227e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:23.451583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0401f-765c-4371-b859-fd591f227e5a None admin] PATH_INFO: `/v3/users/3efa7667c36b4e2581c375766f921376` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:23.453082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0401f-765c-4371-b859-fd591f227e5a None admin] RBAC: Authorizing `identity:get_user(user_id=3efa7667c36b4e2581c375766f921376)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:23.454786 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0401f-765c-4371-b859-fd591f227e5a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:23.457812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 181/370] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:10:23 2022] GET /identity/v3/users/3efa7667c36b4e2581c375766f921376 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:23.459418 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0401f-765c-4371-b859-fd591f227e5a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:23.503271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:23.592093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3316f-2cdf-4d4d-b2f1-a4062ff737bd None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:23.597357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:23.613257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0401f-765c-4371-b859-fd591f227e5a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:23.642560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fef36-ba6d-49f5-ac13-77d1ac9696c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:23.644075 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fef36-ba6d-49f5-ac13-77d1ac9696c2 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:23.644787 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fef36-ba6d-49f5-ac13-77d1ac9696c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:23.645839 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fef36-ba6d-49f5-ac13-77d1ac9696c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:23.647064 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fef36-ba6d-49f5-ac13-77d1ac9696c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:23.647813 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fef36-ba6d-49f5-ac13-77d1ac9696c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:23.650289 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fef36-ba6d-49f5-ac13-77d1ac9696c2 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:23.654137 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6651f0-d0b3-4635-9f75-a3ab597bc4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:23.654781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6651f0-d0b3-4635-9f75-a3ab597bc4bf None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:23.655061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6651f0-d0b3-4635-9f75-a3ab597bc4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:23.655335 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6651f0-d0b3-4635-9f75-a3ab597bc4bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:23.656150 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6651f0-d0b3-4635-9f75-a3ab597bc4bf None admin] RBAC: Adding query filter params (group.id=eec63ee8bad640a39895e1cda8b0f24b, role.id=19db9fb1cb1541ee94673df232bd6952, scope.project.id=66624eb1d31e484ba73e6737100a3e5c) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:23.656458 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6651f0-d0b3-4635-9f75-a3ab597bc4bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:23.659423 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6651f0-d0b3-4635-9f75-a3ab597bc4bf None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:23.663185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 190/371] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:23.664669 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fef36-ba6d-49f5-ac13-77d1ac9696c2 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:23.665808 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 182/372] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 15:10:23 2022] GET /identity/v3/role_assignments?group.id=eec63ee8bad640a39895e1cda8b0f24b&scope.project.id=66624eb1d31e484ba73e6737100a3e5c&role.id=19db9fb1cb1541ee94673df232bd6952 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:23.666623 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6651f0-d0b3-4635-9f75-a3ab597bc4bf None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:23.795799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fef36-ba6d-49f5-ac13-77d1ac9696c2 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:23.799422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:23.861167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6651f0-d0b3-4635-9f75-a3ab597bc4bf None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:23.883296 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33429509-6eab-4eb7-ac00-123b3c44c131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:23.883975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33429509-6eab-4eb7-ac00-123b3c44c131 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:23.884258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33429509-6eab-4eb7-ac00-123b3c44c131 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:23.884533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33429509-6eab-4eb7-ac00-123b3c44c131 None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:23.885739 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33429509-6eab-4eb7-ac00-123b3c44c131 None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:23.887265 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33429509-6eab-4eb7-ac00-123b3c44c131 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:23.889024 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 191/373] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:23 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:23.889872 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33429509-6eab-4eb7-ac00-123b3c44c131 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:23.912619 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca235d5-69fb-4724-9a1a-b3e83de9914e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:23.913299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca235d5-69fb-4724-9a1a-b3e83de9914e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:23.913581 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca235d5-69fb-4724-9a1a-b3e83de9914e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:23.913905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca235d5-69fb-4724-9a1a-b3e83de9914e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:23.914750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca235d5-69fb-4724-9a1a-b3e83de9914e None admin] RBAC: Adding query filter params (role.id=24102e3afb81458ca8d594a2b25b31b2, user.id=3efa7667c36b4e2581c375766f921376, scope.system=all) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:23.915045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca235d5-69fb-4724-9a1a-b3e83de9914e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:23.916529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca235d5-69fb-4724-9a1a-b3e83de9914e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:23.934624 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 183/374] 10.222.0.48 () {56 vars in 1308 bytes} [Mon Aug 1 15:10:23 2022] GET /identity/v3/role_assignments?user.id=3efa7667c36b4e2581c375766f921376&scope.system=all&role.id=24102e3afb81458ca8d594a2b25b31b2 => generated 510 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:23.936052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8790f1-5c50-423e-9a2d-6257a01fa6f2 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:23.936337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8790f1-5c50-423e-9a2d-6257a01fa6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:23.936612 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8790f1-5c50-423e-9a2d-6257a01fa6f2 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:23.943408 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 184/375] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:23.943408 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8790f1-5c50-423e-9a2d-6257a01fa6f2 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:23.958233 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:24.093121 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8790f1-5c50-423e-9a2d-6257a01fa6f2 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:24.103820 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33429509-6eab-4eb7-ac00-123b3c44c131 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:24.135532 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d4836-7be7-437d-8ba7-f5a2feeffd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:24.136482 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d4836-7be7-437d-8ba7-f5a2feeffd3e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:24.136762 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d4836-7be7-437d-8ba7-f5a2feeffd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:24.137032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d4836-7be7-437d-8ba7-f5a2feeffd3e None admin] PATH_INFO: `/v3/groups/eec63ee8bad640a39895e1cda8b0f24b` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:24.138428 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d4836-7be7-437d-8ba7-f5a2feeffd3e None admin] RBAC: Authorizing `identity:get_group(group_id=eec63ee8bad640a39895e1cda8b0f24b)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:24.139968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d4836-7be7-437d-8ba7-f5a2feeffd3e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:24.142144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 185/376] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 15:10:23 2022] GET /identity/v3/groups/eec63ee8bad640a39895e1cda8b0f24b => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:24.143505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44ee0c-daa1-46a6-b859-d05b8a08dec1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:24.143788 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44ee0c-daa1-46a6-b859-d05b8a08dec1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:24.144062 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44ee0c-daa1-46a6-b859-d05b8a08dec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:24.165455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2f44ee0c-daa1-46a6-b859-d05b8a08dec1 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:24.166781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2f44ee0c-daa1-46a6-b859-d05b8a08dec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:24.177918 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96366e-4088-490e-b4f9-4d3169b5f0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:24.178651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96366e-4088-490e-b4f9-4d3169b5f0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:24.178935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96366e-4088-490e-b4f9-4d3169b5f0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:24.179214 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96366e-4088-490e-b4f9-4d3169b5f0e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:24.238035 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:24.253731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 186/377] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:24.254647 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44ee0c-daa1-46a6-b859-d05b8a08dec1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:24.284590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96366e-4088-490e-b4f9-4d3169b5f0e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:24.287999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a96366e-4088-490e-b4f9-4d3169b5f0e4 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:24.290969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:24.395506 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44ee0c-daa1-46a6-b859-d05b8a08dec1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:24.444426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad402f-f1a9-4bf7-bb7e-bed717ebf1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:24.445392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad402f-f1a9-4bf7-bb7e-bed717ebf1ae None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:24.445872 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad402f-f1a9-4bf7-bb7e-bed717ebf1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:24.446313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad402f-f1a9-4bf7-bb7e-bed717ebf1ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:24.447331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad402f-f1a9-4bf7-bb7e-bed717ebf1ae None admin] RBAC: Adding query filter params (group.id=eec63ee8bad640a39895e1cda8b0f24b, role.id=7ae6dccec1244987a5bc7bf5c3991703, scope.project.id=66624eb1d31e484ba73e6737100a3e5c) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:24.447642 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad402f-f1a9-4bf7-bb7e-bed717ebf1ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:24.449111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad402f-f1a9-4bf7-bb7e-bed717ebf1ae None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:24.452186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 192/378] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:24.453187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96366e-4088-490e-b4f9-4d3169b5f0e4 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:24.459030 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 187/379] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 15:10:24 2022] GET /identity/v3/role_assignments?group.id=eec63ee8bad640a39895e1cda8b0f24b&scope.project.id=66624eb1d31e484ba73e6737100a3e5c&role.id=7ae6dccec1244987a5bc7bf5c3991703 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:24.459965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad402f-f1a9-4bf7-bb7e-bed717ebf1ae None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:24.556904 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:24.558937 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:24.589018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad402f-f1a9-4bf7-bb7e-bed717ebf1ae None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:24.600817 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a96366e-4088-490e-b4f9-4d3169b5f0e4 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:24.640355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fbf99-3dbc-415f-8d98-6f92bf4559e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:24.641201 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fbf99-3dbc-415f-8d98-6f92bf4559e7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:24.641544 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fbf99-3dbc-415f-8d98-6f92bf4559e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:24.642433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348fbf99-3dbc-415f-8d98-6f92bf4559e7 None admin] PATH_INFO: `/v3/projects/ef3b9d837c2248a7ae12fa99e632e776` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:24.645620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348fbf99-3dbc-415f-8d98-6f92bf4559e7 None admin] RBAC: Authorizing `identity:get_project(project_id=ef3b9d837c2248a7ae12fa99e632e776)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:24.647484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348fbf99-3dbc-415f-8d98-6f92bf4559e7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:24.649888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 188/380] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:24 2022] GET /identity/v3/projects/ef3b9d837c2248a7ae12fa99e632e776 => generated 344 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:24.652329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c496c-a031-4b8b-89f0-c97bd36822ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:24.653036 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c496c-a031-4b8b-89f0-c97bd36822ed None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:24.653300 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c496c-a031-4b8b-89f0-c97bd36822ed None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:24.653576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c496c-a031-4b8b-89f0-c97bd36822ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:24.659104 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fbf99-3dbc-415f-8d98-6f92bf4559e7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:24.684221 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c496c-a031-4b8b-89f0-c97bd36822ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:24.694901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-f04c496c-a031-4b8b-89f0-c97bd36822ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:24.694901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:24.694901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:24.694901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:24.694901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:24.694901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:24.694901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:24.694901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:24.694901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:24.694901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:24.694901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:24.694901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:24.694901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:24.694901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:24.694901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:24.694901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:24.695902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:24.695902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:24.695902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:24.695902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:24.695902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:24.695902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:24.695902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:24.695902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:24.695902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:24.695902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:24.695902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:24.695902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:24.695902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:24.695902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:24.695902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:24.695902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:24.696681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:24.696681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:24.696681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:24.696681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:24.696681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:24.696681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:24.696681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:24.696681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:24.696681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:24.696681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:24.696681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:24.696681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:24.696681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 193/381] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:24.702567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c496c-a031-4b8b-89f0-c97bd36822ed None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:24.829570 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c496c-a031-4b8b-89f0-c97bd36822ed None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:24.873071 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:24.879895 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a8ff2-0d22-4e3d-a910-9b4d78a5748f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:24.880616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a8ff2-0d22-4e3d-a910-9b4d78a5748f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:24.880962 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a8ff2-0d22-4e3d-a910-9b4d78a5748f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:24.881302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a8ff2-0d22-4e3d-a910-9b4d78a5748f None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:24.882157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a8ff2-0d22-4e3d-a910-9b4d78a5748f None admin] RBAC: Adding query filter params (name=member) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:24.882529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a8ff2-0d22-4e3d-a910-9b4d78a5748f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:24.884274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a8ff2-0d22-4e3d-a910-9b4d78a5748f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:24.895375 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 194/382] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:24.900308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a8ff2-0d22-4e3d-a910-9b4d78a5748f None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:25.003830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348fbf99-3dbc-415f-8d98-6f92bf4559e7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:25.014735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a8ff2-0d22-4e3d-a910-9b4d78a5748f None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:25.054136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edfe2c-41c4-417e-bc67-1d80586e529f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:25.054934 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edfe2c-41c4-417e-bc67-1d80586e529f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:25.055283 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edfe2c-41c4-417e-bc67-1d80586e529f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:25.056196 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7edfe2c-41c4-417e-bc67-1d80586e529f None admin] PATH_INFO: `/v3/groups/eec63ee8bad640a39895e1cda8b0f24b` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:25.057464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edfe2c-41c4-417e-bc67-1d80586e529f None admin] RBAC: Authorizing `identity:get_group(group_id=eec63ee8bad640a39895e1cda8b0f24b)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:25.059109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7edfe2c-41c4-417e-bc67-1d80586e529f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:25.061139 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 195/383] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 15:10:24 2022] GET /identity/v3/groups/eec63ee8bad640a39895e1cda8b0f24b => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:25.066775 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edfe2c-41c4-417e-bc67-1d80586e529f None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:25.123176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e55945-ca16-4252-88ad-53ffd69f8428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:25.123863 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e55945-ca16-4252-88ad-53ffd69f8428 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:25.124148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e55945-ca16-4252-88ad-53ffd69f8428 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:25.124430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e55945-ca16-4252-88ad-53ffd69f8428 None admin] PATH_INFO: `/v3/groups/bcc4adaf726a4c44a47568d4c737a2f8` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:25.139301 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e55945-ca16-4252-88ad-53ffd69f8428 None admin] RBAC: Authorizing `identity:get_group(group_id=bcc4adaf726a4c44a47568d4c737a2f8)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:25.140805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e55945-ca16-4252-88ad-53ffd69f8428 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:25.142766 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 189/384] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 15:10:24 2022] GET /identity/v3/groups/bcc4adaf726a4c44a47568d4c737a2f8 => generated 229 bytes in 484 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:25.145086 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:25.148400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e55945-ca16-4252-88ad-53ffd69f8428 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:25.160089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:25.201030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7edfe2c-41c4-417e-bc67-1d80586e529f None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:25.240203 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd562afd-eb9a-4239-ba94-75f53c844c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:25.240863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd562afd-eb9a-4239-ba94-75f53c844c38 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:25.241200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd562afd-eb9a-4239-ba94-75f53c844c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:25.241528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd562afd-eb9a-4239-ba94-75f53c844c38 None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:25.242942 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd562afd-eb9a-4239-ba94-75f53c844c38 None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:25.244943 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd562afd-eb9a-4239-ba94-75f53c844c38 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:25.247650 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 196/385] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:25 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:25.249864 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796bad8f-b25d-41b3-a41f-269c81b015d9 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:25.250185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796bad8f-b25d-41b3-a41f-269c81b015d9 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:25.250462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796bad8f-b25d-41b3-a41f-269c81b015d9 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:25.256510 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 197/386] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:25.257324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796bad8f-b25d-41b3-a41f-269c81b015d9 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:25.291717 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e55945-ca16-4252-88ad-53ffd69f8428 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:25.343383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f36e7-81e9-4d19-8f6b-8214cdea6a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:25.344562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f36e7-81e9-4d19-8f6b-8214cdea6a13 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:25.345421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f36e7-81e9-4d19-8f6b-8214cdea6a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:25.345910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f36e7-81e9-4d19-8f6b-8214cdea6a13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:25.347115 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f36e7-81e9-4d19-8f6b-8214cdea6a13 None admin] RBAC: Adding query filter params (group.id=bcc4adaf726a4c44a47568d4c737a2f8, role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=ef3b9d837c2248a7ae12fa99e632e776) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:25.347760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f36e7-81e9-4d19-8f6b-8214cdea6a13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:25.350072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f36e7-81e9-4d19-8f6b-8214cdea6a13 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:25.359780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 190/387] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 15:10:25 2022] GET /identity/v3/role_assignments?group.id=bcc4adaf726a4c44a47568d4c737a2f8&scope.project.id=ef3b9d837c2248a7ae12fa99e632e776&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:25.364072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27bdef-da4c-445e-bee3-cc615abf9cdb None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:25.364072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27bdef-da4c-445e-bee3-cc615abf9cdb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:25.364072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27bdef-da4c-445e-bee3-cc615abf9cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:25.390988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3a27bdef-da4c-445e-bee3-cc615abf9cdb None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:25.392204 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3a27bdef-da4c-445e-bee3-cc615abf9cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:25.403823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796bad8f-b25d-41b3-a41f-269c81b015d9 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:25.441044 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:25.449598 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de50bd4-c73d-4654-b000-7c6fc5b96999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:25.450757 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de50bd4-c73d-4654-b000-7c6fc5b96999 None admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:25.451175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de50bd4-c73d-4654-b000-7c6fc5b96999 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:25.451672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de50bd4-c73d-4654-b000-7c6fc5b96999 None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17/groups/eec63ee8bad640a39895e1cda8b0f24b/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:25.454144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de50bd4-c73d-4654-b000-7c6fc5b96999 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab228684d11649dfa97b4acebd727e17, group_id=eec63ee8bad640a39895e1cda8b0f24b, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:25.455921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de50bd4-c73d-4654-b000-7c6fc5b96999 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:25.461071 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:25.473629 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 191/388] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:25.481324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 198/389] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 1 15:10:25 2022] PUT /identity/v3/projects/ab228684d11649dfa97b4acebd727e17/groups/eec63ee8bad640a39895e1cda8b0f24b/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:25.498378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27bdef-da4c-445e-bee3-cc615abf9cdb None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:25.614777 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27bdef-da4c-445e-bee3-cc615abf9cdb None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:25.652903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22712c82-ab20-4319-9991-84469ab8c591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:25.653575 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22712c82-ab20-4319-9991-84469ab8c591 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:25.653872 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22712c82-ab20-4319-9991-84469ab8c591 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:25.654146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22712c82-ab20-4319-9991-84469ab8c591 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:25.667667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22712c82-ab20-4319-9991-84469ab8c591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:25.671776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-22712c82-ab20-4319-9991-84469ab8c591 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:25.671776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:25.671776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:25.671776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:25.671776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:25.671776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:25.671776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:25.671776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:25.671776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:25.671776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:25.671776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:25.671776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:25.671776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:25.671776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:25.671776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:25.671776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:25.672653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:25.672653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:25.672653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:25.672653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:25.672653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:25.672653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:25.672653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:25.672653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:25.672653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:25.672653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:25.672653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:25.672653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:25.672653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:25.672653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:25.672653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:25.672653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:25.673465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:25.673465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:25.673465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:25.673465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:25.673465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:25.673465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:25.673465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:25.673465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:25.673465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:25.673465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:25.673465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:25.673465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:25.677923 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 192/390] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:10:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:25.679609 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de50bd4-c73d-4654-b000-7c6fc5b96999 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:25.818603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de50bd4-c73d-4654-b000-7c6fc5b96999 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:25.858454 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27828e02-a831-45f2-a2f2-84596d9c4295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:25.859828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27828e02-a831-45f2-a2f2-84596d9c4295 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:25.860413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27828e02-a831-45f2-a2f2-84596d9c4295 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:25.861191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27828e02-a831-45f2-a2f2-84596d9c4295 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:25.862507 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27828e02-a831-45f2-a2f2-84596d9c4295 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:25.863097 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27828e02-a831-45f2-a2f2-84596d9c4295 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:25.865126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27828e02-a831-45f2-a2f2-84596d9c4295 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:25.889439 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 199/391] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:25.898948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22712c82-ab20-4319-9991-84469ab8c591 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:25.991812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:26.054017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22712c82-ab20-4319-9991-84469ab8c591 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:26.105054 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05726e6d-0aeb-4cef-abf8-287774b1d7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:26.105642 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05726e6d-0aeb-4cef-abf8-287774b1d7ef None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:26.105985 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05726e6d-0aeb-4cef-abf8-287774b1d7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:26.106260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05726e6d-0aeb-4cef-abf8-287774b1d7ef None admin] PATH_INFO: `/v3/groups/eec63ee8bad640a39895e1cda8b0f24b` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:26.107479 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05726e6d-0aeb-4cef-abf8-287774b1d7ef None admin] RBAC: Authorizing `identity:get_group(group_id=eec63ee8bad640a39895e1cda8b0f24b)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:26.109340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05726e6d-0aeb-4cef-abf8-287774b1d7ef None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:26.111407 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 193/392] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 15:10:25 2022] GET /identity/v3/groups/eec63ee8bad640a39895e1cda8b0f24b => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:26.116930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27828e02-a831-45f2-a2f2-84596d9c4295 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:26.209060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:26.273973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27828e02-a831-45f2-a2f2-84596d9c4295 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:26.321586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b020b-b0f6-4edb-be32-c1ce5a1b4e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:26.322293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b020b-b0f6-4edb-be32-c1ce5a1b4e44 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:26.322617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b020b-b0f6-4edb-be32-c1ce5a1b4e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:26.322848 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b020b-b0f6-4edb-be32-c1ce5a1b4e44 None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:26.324061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b020b-b0f6-4edb-be32-c1ce5a1b4e44 None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:26.325620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b020b-b0f6-4edb-be32-c1ce5a1b4e44 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:26.327404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 200/393] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:26 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:26.333623 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05726e6d-0aeb-4cef-abf8-287774b1d7ef None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:26.470884 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05726e6d-0aeb-4cef-abf8-287774b1d7ef None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:26.513524 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:26.530276 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08afe2-8b6d-41cd-a941-78f01483de3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:26.530980 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08afe2-8b6d-41cd-a941-78f01483de3f None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:26.531259 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08afe2-8b6d-41cd-a941-78f01483de3f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:26.531535 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08afe2-8b6d-41cd-a941-78f01483de3f None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17/groups/eec63ee8bad640a39895e1cda8b0f24b/roles/7ae6dccec1244987a5bc7bf5c3991703` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:26.538684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08afe2-8b6d-41cd-a941-78f01483de3f None admin] RBAC: Authorizing `identity:create_grant(project_id=ab228684d11649dfa97b4acebd727e17, group_id=eec63ee8bad640a39895e1cda8b0f24b, role_id=7ae6dccec1244987a5bc7bf5c3991703)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:26.540193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08afe2-8b6d-41cd-a941-78f01483de3f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:26.566916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 194/394] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 1 15:10:26 2022] PUT /identity/v3/projects/ab228684d11649dfa97b4acebd727e17/groups/eec63ee8bad640a39895e1cda8b0f24b/roles/7ae6dccec1244987a5bc7bf5c3991703 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:26.655337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a5d87-81d4-4e59-b172-fbcc445e8129 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:26.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a5d87-81d4-4e59-b172-fbcc445e8129 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:26.655904 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a5d87-81d4-4e59-b172-fbcc445e8129 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:26.657626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 201/395] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:26.663031 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375c481-2904-43d9-afd3-16a935cc4ffd None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:26.663316 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375c481-2904-43d9-afd3-16a935cc4ffd None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:26.663591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375c481-2904-43d9-afd3-16a935cc4ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:26.690993 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a375c481-2904-43d9-afd3-16a935cc4ffd None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:26.692236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a375c481-2904-43d9-afd3-16a935cc4ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:26.789911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 195/396] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:26.811759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a5d87-81d4-4e59-b172-fbcc445e8129 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:26.954337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a5d87-81d4-4e59-b172-fbcc445e8129 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:26.982026 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:26.997435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4449b8-540e-4a9b-b07b-20a8d15bed39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:26.998123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4449b8-540e-4a9b-b07b-20a8d15bed39 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:26.998405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4449b8-540e-4a9b-b07b-20a8d15bed39 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:26.998682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4449b8-540e-4a9b-b07b-20a8d15bed39 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:27.009315 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4449b8-540e-4a9b-b07b-20a8d15bed39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:27.015201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-3c4449b8-540e-4a9b-b07b-20a8d15bed39 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:27.015201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:27.015201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:27.015201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:27.015201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:27.015201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:27.015201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:27.015201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:27.015201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:27.015201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:27.015201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:27.015201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:27.015201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:27.015201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:27.015201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:27.015201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:27.016186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:27.016186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:27.016186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:27.016186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:27.016186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:27.016186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:27.016186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:27.016186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:27.016186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:27.016186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:27.016186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:27.016186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:27.016186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:27.016186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:27.016186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:27.016186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:27.016976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:27.016976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:27.016976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:27.016976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:27.016976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:27.016976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:27.016976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:27.016976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:27.016976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:27.016976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:27.016976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:27.016976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:27.016976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 202/397] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:27.022692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375c481-2904-43d9-afd3-16a935cc4ffd None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:27.174045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375c481-2904-43d9-afd3-16a935cc4ffd None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:27.203964 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:27.236038 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e931ab5-a9ad-4fe4-a235-7586f7fa2089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:27.237085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e931ab5-a9ad-4fe4-a235-7586f7fa2089 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:27.237601 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e931ab5-a9ad-4fe4-a235-7586f7fa2089 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:27.238101 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e931ab5-a9ad-4fe4-a235-7586f7fa2089 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:27.239029 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e931ab5-a9ad-4fe4-a235-7586f7fa2089 None admin] RBAC: Adding query filter params (name=member) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:27.239523 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e931ab5-a9ad-4fe4-a235-7586f7fa2089 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:27.241283 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e931ab5-a9ad-4fe4-a235-7586f7fa2089 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:27.253945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 196/398] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:27.259276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4449b8-540e-4a9b-b07b-20a8d15bed39 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:27.274345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375decf2-c0ad-4972-8f6c-deceb30107ff None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:27.275047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375decf2-c0ad-4972-8f6c-deceb30107ff None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:27.275519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375decf2-c0ad-4972-8f6c-deceb30107ff None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:27.277344 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 197/399] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:10:27.283099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701db21-b7a3-494d-a04c-20e373b42a01 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:27.283716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701db21-b7a3-494d-a04c-20e373b42a01 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:27.284202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701db21-b7a3-494d-a04c-20e373b42a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:27.304926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d701db21-b7a3-494d-a04c-20e373b42a01 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:27.307302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d701db21-b7a3-494d-a04c-20e373b42a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:27.433801 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 198/400] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:10:27.442625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4449b8-540e-4a9b-b07b-20a8d15bed39 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:27.477810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701db21-b7a3-494d-a04c-20e373b42a01 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:27.493430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e203dae-26f1-4ce2-aa46-98c211523f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:27.494482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e203dae-26f1-4ce2-aa46-98c211523f3f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:27.495755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e203dae-26f1-4ce2-aa46-98c211523f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:27.496921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e203dae-26f1-4ce2-aa46-98c211523f3f None admin] PATH_INFO: `/v3/groups/eec63ee8bad640a39895e1cda8b0f24b` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:27.499984 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e203dae-26f1-4ce2-aa46-98c211523f3f None admin] RBAC: Authorizing `identity:get_group(group_id=eec63ee8bad640a39895e1cda8b0f24b)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:27.504049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e203dae-26f1-4ce2-aa46-98c211523f3f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:27.506006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 203/401] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 15:10:27 2022] GET /identity/v3/groups/eec63ee8bad640a39895e1cda8b0f24b => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:27.512213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e203dae-26f1-4ce2-aa46-98c211523f3f None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:27.578310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:27.637663 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e203dae-26f1-4ce2-aa46-98c211523f3f None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:27.670110 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701db21-b7a3-494d-a04c-20e373b42a01 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:27.675960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5fc0c-e705-4f70-8618-05396dfcfe58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:27.677415 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5fc0c-e705-4f70-8618-05396dfcfe58 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:27.678023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5fc0c-e705-4f70-8618-05396dfcfe58 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:27.678729 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5fc0c-e705-4f70-8618-05396dfcfe58 None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:27.680402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5fc0c-e705-4f70-8618-05396dfcfe58 None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:27.682882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5fc0c-e705-4f70-8618-05396dfcfe58 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:27.684752 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:27.686738 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 204/402] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:27 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:27.698909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5fc0c-e705-4f70-8618-05396dfcfe58 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:27.716173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28898fce-ec2c-4a4f-83dd-63b98ce0b136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:27.716832 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28898fce-ec2c-4a4f-83dd-63b98ce0b136 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:27.717109 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28898fce-ec2c-4a4f-83dd-63b98ce0b136 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:27.717379 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28898fce-ec2c-4a4f-83dd-63b98ce0b136 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:27.728813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28898fce-ec2c-4a4f-83dd-63b98ce0b136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:27.733509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-28898fce-ec2c-4a4f-83dd-63b98ce0b136 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:27.733509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:27.733509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:27.733509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:27.733509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:27.733509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:27.733509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:27.733509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:27.733509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:27.733509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:27.733509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:27.733509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:27.733509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:27.733509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:27.733509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:27.733509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:27.734444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:27.734444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:27.734444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:27.734444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:27.734444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:27.734444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:27.734444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:27.734444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:27.734444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:27.734444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:27.734444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:27.734444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:27.734444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:27.734444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:27.734444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:27.734444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:27.735371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:27.735371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:27.735371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:27.735371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:27.735371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:27.735371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:27.735371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:27.735371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:27.735371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:27.735371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:27.735371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:27.735371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:27.738076 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 199/403] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:10:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:27.743829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28898fce-ec2c-4a4f-83dd-63b98ce0b136 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:27.845684 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5fc0c-e705-4f70-8618-05396dfcfe58 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:27.846490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28898fce-ec2c-4a4f-83dd-63b98ce0b136 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:27.868064 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:27.882083 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d5550-c8b5-4161-ac07-ab340afb1596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:27.882687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d5550-c8b5-4161-ac07-ab340afb1596 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:27.882966 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d5550-c8b5-4161-ac07-ab340afb1596 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:27.883236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d5550-c8b5-4161-ac07-ab340afb1596 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:27.883968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d5550-c8b5-4161-ac07-ab340afb1596 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:27.884260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d5550-c8b5-4161-ac07-ab340afb1596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:27.885663 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d5550-c8b5-4161-ac07-ab340afb1596 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:27.894440 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 200/404] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:27.897654 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e55a0-78fb-44a3-bd6d-5383153a79af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:27.898615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e55a0-78fb-44a3-bd6d-5383153a79af None admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:27.899138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e55a0-78fb-44a3-bd6d-5383153a79af None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:27.899603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e55a0-78fb-44a3-bd6d-5383153a79af None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c/groups/eec63ee8bad640a39895e1cda8b0f24b/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:27.899959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d5550-c8b5-4161-ac07-ab340afb1596 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:27.902119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e55a0-78fb-44a3-bd6d-5383153a79af None admin] RBAC: Authorizing `identity:create_grant(project_id=66624eb1d31e484ba73e6737100a3e5c, group_id=eec63ee8bad640a39895e1cda8b0f24b, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:27.904190 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e55a0-78fb-44a3-bd6d-5383153a79af None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:27.929933 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 205/405] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 1 15:10:27 2022] PUT /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c/groups/eec63ee8bad640a39895e1cda8b0f24b/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:27.931441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc5a9a-b07b-40c1-a6c9-8560ac49f7fc None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:27.931774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc5a9a-b07b-40c1-a6c9-8560ac49f7fc None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:27.932127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:27.932882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc5a9a-b07b-40c1-a6c9-8560ac49f7fc None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:27.939201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 206/406] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:27.941165 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bac5b-525d-4340-96df-e3a78512c6cb None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:27.941453 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bac5b-525d-4340-96df-e3a78512c6cb None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:27.941730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7bac5b-525d-4340-96df-e3a78512c6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:27.961337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-ce7bac5b-525d-4340-96df-e3a78512c6cb None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:27.962687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-ce7bac5b-525d-4340-96df-e3a78512c6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:28.011506 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d5550-c8b5-4161-ac07-ab340afb1596 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:28.041961 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 207/407] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:28.047360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd56c64-7bc9-444f-8c00-6c4f4514ce18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:28.048078 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd56c64-7bc9-444f-8c00-6c4f4514ce18 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:28.048359 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd56c64-7bc9-444f-8c00-6c4f4514ce18 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:28.048666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd56c64-7bc9-444f-8c00-6c4f4514ce18 None admin] PATH_INFO: `/v3/groups/eec63ee8bad640a39895e1cda8b0f24b` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:28.049984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd56c64-7bc9-444f-8c00-6c4f4514ce18 None admin] RBAC: Authorizing `identity:get_group(group_id=eec63ee8bad640a39895e1cda8b0f24b)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:28.051563 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd56c64-7bc9-444f-8c00-6c4f4514ce18 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:28.053586 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 201/408] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 15:10:27 2022] GET /identity/v3/groups/eec63ee8bad640a39895e1cda8b0f24b => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:28.059496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bac5b-525d-4340-96df-e3a78512c6cb None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:28.066709 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd56c64-7bc9-444f-8c00-6c4f4514ce18 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:28.201413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7bac5b-525d-4340-96df-e3a78512c6cb None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:28.211484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:28.239627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:28.243405 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd56c64-7bc9-444f-8c00-6c4f4514ce18 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:28.244049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337227a-c697-42e8-b702-b9750637044b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:28.244879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337227a-c697-42e8-b702-b9750637044b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:28.245337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337227a-c697-42e8-b702-b9750637044b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:28.245753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337227a-c697-42e8-b702-b9750637044b None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:28.247474 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337227a-c697-42e8-b702-b9750637044b None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:28.250184 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337227a-c697-42e8-b702-b9750637044b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:28.252468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 208/409] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:28 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:28.254053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f199650-6c8b-461a-b8b9-a24cb6173bb4 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:28.254474 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f199650-6c8b-461a-b8b9-a24cb6173bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:28.254887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f199650-6c8b-461a-b8b9-a24cb6173bb4 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:28.260455 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 209/410] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:28.265593 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f199650-6c8b-461a-b8b9-a24cb6173bb4 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:28.285911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42683b-dab8-4441-8a80-0952b5539f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:28.286722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42683b-dab8-4441-8a80-0952b5539f88 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:28.287141 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42683b-dab8-4441-8a80-0952b5539f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:28.287545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42683b-dab8-4441-8a80-0952b5539f88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:28.298560 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42683b-dab8-4441-8a80-0952b5539f88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:28.304530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-9d42683b-dab8-4441-8a80-0952b5539f88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:10:28.304530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:28.304530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:28.304530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:28.304530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:28.304530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:28.304530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:28.304530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:28.304530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:28.304530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:28.304530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:28.304530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:28.304530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:28.304530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:28.304530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:28.304530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:28.305470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:28.305470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:28.305470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:28.305470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:28.305470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:28.305470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:28.305470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:28.305470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:28.305470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:28.305470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:28.305470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:28.305470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:28.305470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:28.305470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:28.305470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:28.305470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:28.306334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:28.306334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:28.306334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:28.306334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:28.306334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:28.306334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:28.306334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:28.306334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:28.306334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:28.306334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:28.306334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:10:28.306334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:28.306334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 202/411] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:10:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:28.307981 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4d746-1933-49ae-bd1b-7b79a3b43f92 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:28.308390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4d746-1933-49ae-bd1b-7b79a3b43f92 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:28.308803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4d746-1933-49ae-bd1b-7b79a3b43f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:28.329326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4fe4d746-1933-49ae-bd1b-7b79a3b43f92 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:28.330819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4fe4d746-1933-49ae-bd1b-7b79a3b43f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:28.381890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f199650-6c8b-461a-b8b9-a24cb6173bb4 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:28.409808 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 203/412] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:28.409808 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4d746-1933-49ae-bd1b-7b79a3b43f92 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:28.437619 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720e3e9-e307-46d8-9a47-33d1faa3c362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:28.438772 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720e3e9-e307-46d8-9a47-33d1faa3c362 None admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:28.439280 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720e3e9-e307-46d8-9a47-33d1faa3c362 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:28.439751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720e3e9-e307-46d8-9a47-33d1faa3c362 None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c/groups/eec63ee8bad640a39895e1cda8b0f24b/roles/7ae6dccec1244987a5bc7bf5c3991703` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:28.442377 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c720e3e9-e307-46d8-9a47-33d1faa3c362 None admin] RBAC: Authorizing `identity:create_grant(project_id=66624eb1d31e484ba73e6737100a3e5c, group_id=eec63ee8bad640a39895e1cda8b0f24b, role_id=7ae6dccec1244987a5bc7bf5c3991703)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:28.444040 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c720e3e9-e307-46d8-9a47-33d1faa3c362 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:28.466018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 210/413] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 1 15:10:28 2022] PUT /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c/groups/eec63ee8bad640a39895e1cda8b0f24b/roles/7ae6dccec1244987a5bc7bf5c3991703 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:28.467676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720e3e9-e307-46d8-9a47-33d1faa3c362 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:28.507945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:28.521019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:28.536366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4d746-1933-49ae-bd1b-7b79a3b43f92 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:28.587357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e108caa-7aae-47c3-85ce-de3f117fc529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:28.588645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e108caa-7aae-47c3-85ce-de3f117fc529 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:28.589125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e108caa-7aae-47c3-85ce-de3f117fc529 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:28.589888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e108caa-7aae-47c3-85ce-de3f117fc529 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:28.591375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e108caa-7aae-47c3-85ce-de3f117fc529 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:28.592141 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e108caa-7aae-47c3-85ce-de3f117fc529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:28.594536 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e108caa-7aae-47c3-85ce-de3f117fc529 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:28.602642 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720e3e9-e307-46d8-9a47-33d1faa3c362 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:28.607449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 204/414] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:28.613726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e108caa-7aae-47c3-85ce-de3f117fc529 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:28.650349 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ffcee-79e9-44c8-ad90-9118374ed5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:28.651704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ffcee-79e9-44c8-ad90-9118374ed5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:28.652186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ffcee-79e9-44c8-ad90-9118374ed5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:28.653153 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ffcee-79e9-44c8-ad90-9118374ed5d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:28.666912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ffcee-79e9-44c8-ad90-9118374ed5d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:28.673447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-785ffcee-79e9-44c8-ad90-9118374ed5d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:28.673447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:28.673447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:28.673447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:28.673447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:28.673447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:28.673447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:28.673447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:28.673447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:28.673447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:28.673447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:28.673447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:28.673447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:28.673447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:28.673447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:28.673447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:28.674502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:28.674502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:28.674502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:28.674502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:28.674502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:28.674502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:28.674502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:28.674502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:28.674502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:28.674502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:28.674502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:28.674502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:28.674502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:28.674502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:28.674502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:28.674502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:28.675318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:28.675318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:28.675318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:28.675318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:28.675318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:28.675318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:28.675318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:28.675318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:28.675318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:28.675318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:28.675318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:28.675318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:28.678347 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 211/415] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:28.684701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ffcee-79e9-44c8-ad90-9118374ed5d6 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:28.805920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e108caa-7aae-47c3-85ce-de3f117fc529 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:28.810236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:28.819147 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ffcee-79e9-44c8-ad90-9118374ed5d6 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:28.848652 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2039ca8-2208-4279-8766-f709638bbd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:28.849325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039ca8-2208-4279-8766-f709638bbd2a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:28.849617 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039ca8-2208-4279-8766-f709638bbd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:28.849921 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039ca8-2208-4279-8766-f709638bbd2a None admin] PATH_INFO: `/v3/groups/bcc4adaf726a4c44a47568d4c737a2f8` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:28.851156 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2039ca8-2208-4279-8766-f709638bbd2a None admin] RBAC: Authorizing `identity:get_group(group_id=bcc4adaf726a4c44a47568d4c737a2f8)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:28.852699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2039ca8-2208-4279-8766-f709638bbd2a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:28.854741 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 205/416] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 15:10:28 2022] GET /identity/v3/groups/bcc4adaf726a4c44a47568d4c737a2f8 => generated 229 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:28.868872 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2039ca8-2208-4279-8766-f709638bbd2a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:28.871480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa658b3-599c-432c-9092-537e6f4c0b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:28.872987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa658b3-599c-432c-9092-537e6f4c0b21 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:28.873468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa658b3-599c-432c-9092-537e6f4c0b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:28.873999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa658b3-599c-432c-9092-537e6f4c0b21 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:28.874943 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa658b3-599c-432c-9092-537e6f4c0b21 None admin] RBAC: Adding query filter params (name=member) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:28.875457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa658b3-599c-432c-9092-537e6f4c0b21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:28.877140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa658b3-599c-432c-9092-537e6f4c0b21 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:28.888115 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 212/417] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:28.895198 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa658b3-599c-432c-9092-537e6f4c0b21 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:28.910146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:29.018356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa658b3-599c-432c-9092-537e6f4c0b21 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:29.058965 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762abaf-3888-42dd-bfc8-bf80d3b389e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:29.060137 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762abaf-3888-42dd-bfc8-bf80d3b389e5 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:29.060623 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762abaf-3888-42dd-bfc8-bf80d3b389e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:29.061259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762abaf-3888-42dd-bfc8-bf80d3b389e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:29.077288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2039ca8-2208-4279-8766-f709638bbd2a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:29.096781 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:29.115537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f03b78-2ee6-4621-b43e-d8ad52c21320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:29.116338 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f03b78-2ee6-4621-b43e-d8ad52c21320 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:29.116692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f03b78-2ee6-4621-b43e-d8ad52c21320 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:29.117055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f03b78-2ee6-4621-b43e-d8ad52c21320 None admin] PATH_INFO: `/v3/projects/ef3b9d837c2248a7ae12fa99e632e776` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:29.118500 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f03b78-2ee6-4621-b43e-d8ad52c21320 None admin] RBAC: Authorizing `identity:get_project(project_id=ef3b9d837c2248a7ae12fa99e632e776)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:29.120165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f03b78-2ee6-4621-b43e-d8ad52c21320 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:29.122612 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 206/418] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:28 2022] GET /identity/v3/projects/ef3b9d837c2248a7ae12fa99e632e776 => generated 344 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:29.127961 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fb28f-40cd-44df-bede-96c3f217e70a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:29.127961 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fb28f-40cd-44df-bede-96c3f217e70a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:29.127961 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fb28f-40cd-44df-bede-96c3f217e70a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:29.131509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 207/419] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:29.132281 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fb28f-40cd-44df-bede-96c3f217e70a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:29.173083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762abaf-3888-42dd-bfc8-bf80d3b389e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:29.176120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762abaf-3888-42dd-bfc8-bf80d3b389e5 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:29.258195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:29.278828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fb28f-40cd-44df-bede-96c3f217e70a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:29.319041 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 213/420] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:10:29.321006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864d2c6-c982-471c-8fad-99403d2715dc None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:29.321540 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864d2c6-c982-471c-8fad-99403d2715dc None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:29.322041 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864d2c6-c982-471c-8fad-99403d2715dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:29.326246 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08ed1d-1c67-4992-a5f7-fc208081c710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:29.326934 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08ed1d-1c67-4992-a5f7-fc208081c710 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:29.327211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08ed1d-1c67-4992-a5f7-fc208081c710 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:29.327487 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08ed1d-1c67-4992-a5f7-fc208081c710 None admin] PATH_INFO: `/v3/projects/ef3b9d837c2248a7ae12fa99e632e776/groups/bcc4adaf726a4c44a47568d4c737a2f8/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:29.329644 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08ed1d-1c67-4992-a5f7-fc208081c710 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef3b9d837c2248a7ae12fa99e632e776, group_id=bcc4adaf726a4c44a47568d4c737a2f8, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:29.332496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08ed1d-1c67-4992-a5f7-fc208081c710 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:29.343257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-9864d2c6-c982-471c-8fad-99403d2715dc None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:29.344873 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9864d2c6-c982-471c-8fad-99403d2715dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:29.347710 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 208/421] 10.222.0.48 () {58 vars in 1442 bytes} [Mon Aug 1 15:10:29 2022] PUT /identity/v3/projects/ef3b9d837c2248a7ae12fa99e632e776/groups/bcc4adaf726a4c44a47568d4c737a2f8/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:29.349655 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08ed1d-1c67-4992-a5f7-fc208081c710 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:29.410675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:29.443673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 214/422] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:29.478280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08ed1d-1c67-4992-a5f7-fc208081c710 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:29.484115 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864d2c6-c982-471c-8fad-99403d2715dc None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:29.520678 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e8925-2ba2-4533-b4fc-9d24ae7d07e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:29.521347 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e8925-2ba2-4533-b4fc-9d24ae7d07e2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:29.521627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e8925-2ba2-4533-b4fc-9d24ae7d07e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:29.521924 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e8925-2ba2-4533-b4fc-9d24ae7d07e2 None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:29.523392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e8925-2ba2-4533-b4fc-9d24ae7d07e2 None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:29.525373 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e8925-2ba2-4533-b4fc-9d24ae7d07e2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:29.527478 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 209/423] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:29 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:29.536025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e8925-2ba2-4533-b4fc-9d24ae7d07e2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:29.557240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:29.654879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864d2c6-c982-471c-8fad-99403d2715dc None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:29.684198 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e8925-2ba2-4533-b4fc-9d24ae7d07e2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:29.694535 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc545d6c-ade6-4a18-af2a-17d1eae3f366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:29.695518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc545d6c-ade6-4a18-af2a-17d1eae3f366 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:29.696027 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc545d6c-ade6-4a18-af2a-17d1eae3f366 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:29.696585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc545d6c-ade6-4a18-af2a-17d1eae3f366 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:29.708977 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc545d6c-ade6-4a18-af2a-17d1eae3f366 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:29.715178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-cc545d6c-ade6-4a18-af2a-17d1eae3f366 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:29.715178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:29.715178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:29.715178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:29.715178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:29.715178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:29.715178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:29.715178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:29.715178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:29.715178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:29.715178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:29.715178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:29.715178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:29.715178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:29.715178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:29.715178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:29.716077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:29.716077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:29.716077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:29.716077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:29.716077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:29.716077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:29.716077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:29.716077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:29.716077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:29.716077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:29.716077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:29.716077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:29.716077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:29.716077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:29.716077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:29.716077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:29.716914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:29.716914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:29.716914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:29.716914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:29.716914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:29.716914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:29.716914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:29.716914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:29.716914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:29.716914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:29.716914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:29.716914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:29.719567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 215/424] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:10:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:29.724157 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7d66c-7be1-4c2a-a788-5e9e6b319b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:29.724824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7d66c-7be1-4c2a-a788-5e9e6b319b37 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:29.725106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7d66c-7be1-4c2a-a788-5e9e6b319b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:29.725378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7d66c-7be1-4c2a-a788-5e9e6b319b37 None admin] PATH_INFO: `/v3/groups/eec63ee8bad640a39895e1cda8b0f24b` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:29.726381 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc545d6c-ade6-4a18-af2a-17d1eae3f366 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:29.727241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7d66c-7be1-4c2a-a788-5e9e6b319b37 None admin] RBAC: Authorizing `identity:get_group(group_id=eec63ee8bad640a39895e1cda8b0f24b)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:29.728712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7d66c-7be1-4c2a-a788-5e9e6b319b37 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:29.730703 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 210/425] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 15:10:29 2022] GET /identity/v3/groups/eec63ee8bad640a39895e1cda8b0f24b => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:29.742845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7d66c-7be1-4c2a-a788-5e9e6b319b37 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:29.844462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:29.847689 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:29.882274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7d66c-7be1-4c2a-a788-5e9e6b319b37 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:29.921196 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc545d6c-ade6-4a18-af2a-17d1eae3f366 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:29.947955 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f09f6-84b7-4eec-b69a-ef91e1cf3c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:29.948595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f09f6-84b7-4eec-b69a-ef91e1cf3c82 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:29.948877 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f09f6-84b7-4eec-b69a-ef91e1cf3c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:29.949159 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f09f6-84b7-4eec-b69a-ef91e1cf3c82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:29.950002 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f09f6-84b7-4eec-b69a-ef91e1cf3c82 None admin] RBAC: Adding query filter params (group.id=eec63ee8bad640a39895e1cda8b0f24b, role.id=19db9fb1cb1541ee94673df232bd6952, scope.project.id=ab228684d11649dfa97b4acebd727e17) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:29.950313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f09f6-84b7-4eec-b69a-ef91e1cf3c82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:29.954557 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f09f6-84b7-4eec-b69a-ef91e1cf3c82 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:29.963998 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 211/426] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 15:10:29 2022] GET /identity/v3/role_assignments?group.id=eec63ee8bad640a39895e1cda8b0f24b&scope.project.id=ab228684d11649dfa97b4acebd727e17&role.id=19db9fb1cb1541ee94673df232bd6952 => generated 611 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:29.965964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc00313-71dd-4bf5-b332-06f48469a0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:29.966831 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc00313-71dd-4bf5-b332-06f48469a0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:29.967330 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc00313-71dd-4bf5-b332-06f48469a0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:29.967801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc00313-71dd-4bf5-b332-06f48469a0d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:29.968756 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc00313-71dd-4bf5-b332-06f48469a0d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:29.969277 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc00313-71dd-4bf5-b332-06f48469a0d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:29.971018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc00313-71dd-4bf5-b332-06f48469a0d2 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:29.981448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 216/427] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:29.987559 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f09f6-84b7-4eec-b69a-ef91e1cf3c82 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:30.140305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f09f6-84b7-4eec-b69a-ef91e1cf3c82 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:30.179276 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc47d2-3567-43c7-b0f0-fb63b6f209d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:30.180256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc47d2-3567-43c7-b0f0-fb63b6f209d4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:30.180733 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc47d2-3567-43c7-b0f0-fb63b6f209d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:30.181239 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc47d2-3567-43c7-b0f0-fb63b6f209d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:30.201649 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:30.269250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafc47d2-3567-43c7-b0f0-fb63b6f209d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:30.270873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafc47d2-3567-43c7-b0f0-fb63b6f209d4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:30.277768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6927f3e-0364-4faa-8df8-10de64d2b20d None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:30.278113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6927f3e-0364-4faa-8df8-10de64d2b20d None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:30.278391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6927f3e-0364-4faa-8df8-10de64d2b20d None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:30.279984 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 217/428] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:30.285497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7d3b4-d6ca-42ec-8cd7-8881b254fe68 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:30.285626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7d3b4-d6ca-42ec-8cd7-8881b254fe68 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:30.285862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7d3b4-d6ca-42ec-8cd7-8881b254fe68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:30.305202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-a3b7d3b4-d6ca-42ec-8cd7-8881b254fe68 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:30.306425 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-a3b7d3b4-d6ca-42ec-8cd7-8881b254fe68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:30.395625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 218/429] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:10:30.422310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7d3b4-d6ca-42ec-8cd7-8881b254fe68 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:30.431167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 212/430] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:30.438506 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc47d2-3567-43c7-b0f0-fb63b6f209d4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:30.472709 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:30.490213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:30.548128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc47d2-3567-43c7-b0f0-fb63b6f209d4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:30.549314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7d3b4-d6ca-42ec-8cd7-8881b254fe68 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:30.588364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae355ab-e4e0-4bdc-80cd-ad454ec5068e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:30.589392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae355ab-e4e0-4bdc-80cd-ad454ec5068e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:30.589912 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae355ab-e4e0-4bdc-80cd-ad454ec5068e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:30.590369 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae355ab-e4e0-4bdc-80cd-ad454ec5068e None admin] PATH_INFO: `/v3/projects/ab228684d11649dfa97b4acebd727e17` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:30.590534 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c92c9-71bb-4bc1-9b6e-c6d8bf3cb9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:30.591167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c92c9-71bb-4bc1-9b6e-c6d8bf3cb9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:30.591447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c92c9-71bb-4bc1-9b6e-c6d8bf3cb9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:30.591723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c92c9-71bb-4bc1-9b6e-c6d8bf3cb9a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:30.592469 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae355ab-e4e0-4bdc-80cd-ad454ec5068e None admin] RBAC: Authorizing `identity:get_project(project_id=ab228684d11649dfa97b4acebd727e17)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:30.594465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae355ab-e4e0-4bdc-80cd-ad454ec5068e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:30.596832 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 213/431] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:30 2022] GET /identity/v3/projects/ab228684d11649dfa97b4acebd727e17 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:30.602123 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae355ab-e4e0-4bdc-80cd-ad454ec5068e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:30.602801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c92c9-71bb-4bc1-9b6e-c6d8bf3cb9a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:30.609462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-f36c92c9-71bb-4bc1-9b6e-c6d8bf3cb9a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:30.609462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:30.609462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:30.609462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:30.609462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:30.609462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:30.609462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:30.609462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:30.609462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:30.609462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:30.609462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:30.609462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:30.609462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:30.609462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:30.609462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:30.609462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:30.610408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:30.610408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:30.610408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:30.610408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:30.610408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:30.610408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:30.610408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:30.610408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:30.610408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:30.610408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:30.610408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:30.610408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:30.610408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:30.610408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:30.610408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:30.610408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:30.611258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:30.611258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:30.611258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:30.611258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:30.611258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:30.611258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:30.611258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:30.611258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:30.611258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:30.611258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:30.611258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:10:30.611258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:30.611258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 219/432] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:30.619536 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c92c9-71bb-4bc1-9b6e-c6d8bf3cb9a7 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:30.724651 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae355ab-e4e0-4bdc-80cd-ad454ec5068e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:30.726311 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:30.730370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c92c9-71bb-4bc1-9b6e-c6d8bf3cb9a7 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:30.758325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:30.771875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae59e1a-3e09-41e0-a25f-ead93ae4fb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:30.772604 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae59e1a-3e09-41e0-a25f-ead93ae4fb29 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:30.772770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43089737-2bd0-4b13-a349-ce9cf716ccb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:30.772888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae59e1a-3e09-41e0-a25f-ead93ae4fb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:30.773147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae59e1a-3e09-41e0-a25f-ead93ae4fb29 None admin] PATH_INFO: `/v3/groups/eec63ee8bad640a39895e1cda8b0f24b` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:30.773661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43089737-2bd0-4b13-a349-ce9cf716ccb8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:30.774078 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43089737-2bd0-4b13-a349-ce9cf716ccb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:30.774353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae59e1a-3e09-41e0-a25f-ead93ae4fb29 None admin] RBAC: Authorizing `identity:get_group(group_id=eec63ee8bad640a39895e1cda8b0f24b)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:30.774522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43089737-2bd0-4b13-a349-ce9cf716ccb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:30.775365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43089737-2bd0-4b13-a349-ce9cf716ccb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:30.775767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43089737-2bd0-4b13-a349-ce9cf716ccb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:30.775878 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae59e1a-3e09-41e0-a25f-ead93ae4fb29 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:30.777320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43089737-2bd0-4b13-a349-ce9cf716ccb8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:30.783271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 214/433] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 15:10:30 2022] GET /identity/v3/groups/eec63ee8bad640a39895e1cda8b0f24b => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:30.785256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae59e1a-3e09-41e0-a25f-ead93ae4fb29 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:30.788669 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 220/434] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:10:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:30.794934 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43089737-2bd0-4b13-a349-ce9cf716ccb8 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:30.916046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43089737-2bd0-4b13-a349-ce9cf716ccb8 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:30.925950 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae59e1a-3e09-41e0-a25f-ead93ae4fb29 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:30.952670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb0b28-4884-4781-a845-f9ab701f4e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:30.953392 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb0b28-4884-4781-a845-f9ab701f4e84 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:30.953620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb0b28-4884-4781-a845-f9ab701f4e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:30.953946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb0b28-4884-4781-a845-f9ab701f4e84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:30.975504 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1402b24-b239-4dfc-bbd7-3ee90f543a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:30.976507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1402b24-b239-4dfc-bbd7-3ee90f543a60 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:30.976975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1402b24-b239-4dfc-bbd7-3ee90f543a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:30.977429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1402b24-b239-4dfc-bbd7-3ee90f543a60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:30.978971 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1402b24-b239-4dfc-bbd7-3ee90f543a60 None admin] RBAC: Adding query filter params (group.id=eec63ee8bad640a39895e1cda8b0f24b, role.id=7ae6dccec1244987a5bc7bf5c3991703, scope.project.id=ab228684d11649dfa97b4acebd727e17) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:30.979689 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1402b24-b239-4dfc-bbd7-3ee90f543a60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:30.982178 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1402b24-b239-4dfc-bbd7-3ee90f543a60 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:30.985493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:30.999453 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 215/435] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 15:10:30 2022] GET /identity/v3/role_assignments?group.id=eec63ee8bad640a39895e1cda8b0f24b&scope.project.id=ab228684d11649dfa97b4acebd727e17&role.id=7ae6dccec1244987a5bc7bf5c3991703 => generated 611 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:31.033396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb0b28-4884-4781-a845-f9ab701f4e84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:31.035134 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb0b28-4884-4781-a845-f9ab701f4e84 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:31.120943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc92ee5-c726-407b-9038-01a59195f6d7 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:31.121228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc92ee5-c726-407b-9038-01a59195f6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:31.121512 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc92ee5-c726-407b-9038-01a59195f6d7 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:31.123190 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 216/436] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:31.128579 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a2c37-7e24-4a76-b426-3dc346631493 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:31.128863 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a2c37-7e24-4a76-b426-3dc346631493 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:31.129140 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a2c37-7e24-4a76-b426-3dc346631493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:31.148468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-035a2c37-7e24-4a76-b426-3dc346631493 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:31.149737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-035a2c37-7e24-4a76-b426-3dc346631493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:31.155590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 221/437] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:10:31.161366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb0b28-4884-4781-a845-f9ab701f4e84 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:31.203081 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:31.233760 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:31.237468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 217/438] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:10:31.258532 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a2c37-7e24-4a76-b426-3dc346631493 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:31.276855 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb0b28-4884-4781-a845-f9ab701f4e84 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:31.321272 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac1dc8-a983-4364-9579-0315d1adc0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:31.321959 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac1dc8-a983-4364-9579-0315d1adc0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:31.322239 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac1dc8-a983-4364-9579-0315d1adc0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:31.322515 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac1dc8-a983-4364-9579-0315d1adc0e6 None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:31.323842 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ac1dc8-a983-4364-9579-0315d1adc0e6 None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:31.326630 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ac1dc8-a983-4364-9579-0315d1adc0e6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:31.328529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 222/439] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:31 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:31.335829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac1dc8-a983-4364-9579-0315d1adc0e6 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:31.376301 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a2c37-7e24-4a76-b426-3dc346631493 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:31.440362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2f1cb-e087-4559-b591-fe35af62d98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:31.441041 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2f1cb-e087-4559-b591-fe35af62d98a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:31.441322 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2f1cb-e087-4559-b591-fe35af62d98a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:31.441609 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2f1cb-e087-4559-b591-fe35af62d98a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:31.459272 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2f1cb-e087-4559-b591-fe35af62d98a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:31.464319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-68f2f1cb-e087-4559-b591-fe35af62d98a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:31.464319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:31.464319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:31.464319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:31.464319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:31.464319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:31.464319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:31.464319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:31.464319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:31.464319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:31.464319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:31.464319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:31.464319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:31.464319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:31.464319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:31.464319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:31.465401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:31.465401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:31.465401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:31.465401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:31.465401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:31.465401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:31.465401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:31.465401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:31.465401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:31.465401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:31.465401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:31.465401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:31.465401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:31.465401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:31.465401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:31.465401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:31.466429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:31.466429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:31.466429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:31.466429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:31.466429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:31.466429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:31.466429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:31.466429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:31.466429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:31.466429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:31.466429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:31.466429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:31.466429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 218/440] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:10:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:31.472029 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2f1cb-e087-4559-b591-fe35af62d98a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:31.486548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac1dc8-a983-4364-9579-0315d1adc0e6 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:31.516413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:31.521362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:31.536938 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a17ae-5c70-4463-9c5f-ad99e5e3b710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:31.540673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a17ae-5c70-4463-9c5f-ad99e5e3b710 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:31.540673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a17ae-5c70-4463-9c5f-ad99e5e3b710 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:31.540673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a17ae-5c70-4463-9c5f-ad99e5e3b710 None admin] PATH_INFO: `/v3/groups/eec63ee8bad640a39895e1cda8b0f24b` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:31.542124 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a17ae-5c70-4463-9c5f-ad99e5e3b710 None admin] RBAC: Authorizing `identity:get_group(group_id=eec63ee8bad640a39895e1cda8b0f24b)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:31.543836 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a17ae-5c70-4463-9c5f-ad99e5e3b710 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:31.545700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 223/441] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 15:10:31 2022] GET /identity/v3/groups/eec63ee8bad640a39895e1cda8b0f24b => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:31.551418 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a17ae-5c70-4463-9c5f-ad99e5e3b710 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:31.594050 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2f1cb-e087-4559-b591-fe35af62d98a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:31.640566 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d86116-e8de-4446-bb1f-45c4b8fff5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:31.641224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d86116-e8de-4446-bb1f-45c4b8fff5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:31.641492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d86116-e8de-4446-bb1f-45c4b8fff5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:31.641744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d86116-e8de-4446-bb1f-45c4b8fff5a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:31.642759 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d86116-e8de-4446-bb1f-45c4b8fff5a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:31.643060 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d86116-e8de-4446-bb1f-45c4b8fff5a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:31.644454 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d86116-e8de-4446-bb1f-45c4b8fff5a6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:31.660454 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 219/442] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:31.661394 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d86116-e8de-4446-bb1f-45c4b8fff5a6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:31.673761 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a17ae-5c70-4463-9c5f-ad99e5e3b710 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:31.709295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e1700-d682-42fe-aa03-92c579ea2e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:31.709951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e1700-d682-42fe-aa03-92c579ea2e9a None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:31.710234 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e1700-d682-42fe-aa03-92c579ea2e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:31.710506 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e1700-d682-42fe-aa03-92c579ea2e9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:31.711316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e1700-d682-42fe-aa03-92c579ea2e9a None admin] RBAC: Adding query filter params (group.id=eec63ee8bad640a39895e1cda8b0f24b, role.id=19db9fb1cb1541ee94673df232bd6952, scope.project.id=66624eb1d31e484ba73e6737100a3e5c) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:31.711632 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e1700-d682-42fe-aa03-92c579ea2e9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:31.713159 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e1700-d682-42fe-aa03-92c579ea2e9a None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:31.719330 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 224/443] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 15:10:31 2022] GET /identity/v3/role_assignments?group.id=eec63ee8bad640a39895e1cda8b0f24b&scope.project.id=66624eb1d31e484ba73e6737100a3e5c&role.id=19db9fb1cb1541ee94673df232bd6952 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:31.770055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:31.775301 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d86116-e8de-4446-bb1f-45c4b8fff5a6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:31.815792 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e652c7d-8547-437b-b55d-3fb82df7caf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:31.816426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e652c7d-8547-437b-b55d-3fb82df7caf9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:31.816751 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e652c7d-8547-437b-b55d-3fb82df7caf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:31.817026 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e652c7d-8547-437b-b55d-3fb82df7caf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:31.887654 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb862640-efcd-4665-9f59-af8aa8d2ef07 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:31.887941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb862640-efcd-4665-9f59-af8aa8d2ef07 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:31.888216 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb862640-efcd-4665-9f59-af8aa8d2ef07 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:31.889589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 225/444] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:10:31.894863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee930a7-92be-478a-9eed-386cf6c189fa None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:31.895143 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee930a7-92be-478a-9eed-386cf6c189fa None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:31.895419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee930a7-92be-478a-9eed-386cf6c189fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:31.900589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e652c7d-8547-437b-b55d-3fb82df7caf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:31.902102 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e652c7d-8547-437b-b55d-3fb82df7caf9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:31.919707 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-cee930a7-92be-478a-9eed-386cf6c189fa None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:31.920950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-cee930a7-92be-478a-9eed-386cf6c189fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:31.989369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:32.003429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 226/445] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:32.014057 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:32.020371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 220/446] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:32.024958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee930a7-92be-478a-9eed-386cf6c189fa None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:32.026359 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e652c7d-8547-437b-b55d-3fb82df7caf9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:32.133596 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee930a7-92be-478a-9eed-386cf6c189fa None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:32.133915 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e652c7d-8547-437b-b55d-3fb82df7caf9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:32.166756 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff0f54-91c9-4a79-8fda-5a0420d9980c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:32.166948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc079968-1692-493f-a93e-c99b0d205c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:32.167396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff0f54-91c9-4a79-8fda-5a0420d9980c None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:32.167488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc079968-1692-493f-a93e-c99b0d205c78 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:32.167670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff0f54-91c9-4a79-8fda-5a0420d9980c None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:32.167760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc079968-1692-493f-a93e-c99b0d205c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:32.168130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc079968-1692-493f-a93e-c99b0d205c78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:32.168285 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff0f54-91c9-4a79-8fda-5a0420d9980c None admin] PATH_INFO: `/v3/projects/66624eb1d31e484ba73e6737100a3e5c` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:32.169266 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff0f54-91c9-4a79-8fda-5a0420d9980c None admin] RBAC: Authorizing `identity:get_project(project_id=66624eb1d31e484ba73e6737100a3e5c)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:32.170747 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff0f54-91c9-4a79-8fda-5a0420d9980c None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:32.172416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 227/447] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:32 2022] GET /identity/v3/projects/66624eb1d31e484ba73e6737100a3e5c => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:32.177923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff0f54-91c9-4a79-8fda-5a0420d9980c None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:32.178216 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc079968-1692-493f-a93e-c99b0d205c78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:32.183240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-bc079968-1692-493f-a93e-c99b0d205c78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:10:32.183240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:32.183240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:32.183240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:32.183240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:32.183240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:32.183240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:32.183240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:32.183240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:32.183240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:32.183240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:32.183240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:32.183240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:32.183240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:32.183240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:32.183240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:32.184328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:32.184328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:32.184328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:32.184328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:32.184328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:32.184328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:32.184328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:32.184328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:32.184328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:32.184328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:32.184328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:32.184328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:32.184328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:32.184328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:32.184328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:32.184328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:32.185258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:32.185258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:32.185258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:32.185258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:32.185258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:32.185258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:32.185258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:32.185258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:32.185258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:32.185258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:32.185258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:10:32.185258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:32.185258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 221/448] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:10:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:32.190483 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc079968-1692-493f-a93e-c99b0d205c78 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:32.237258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:32.251367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:32.280457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff0f54-91c9-4a79-8fda-5a0420d9980c None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:32.293642 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc079968-1692-493f-a93e-c99b0d205c78 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:32.315294 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5badbf14-c30d-4dc1-adc0-cff47464a9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:32.316025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badbf14-c30d-4dc1-adc0-cff47464a9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:32.316305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badbf14-c30d-4dc1-adc0-cff47464a9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:32.316581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badbf14-c30d-4dc1-adc0-cff47464a9c0 None admin] PATH_INFO: `/v3/groups/eec63ee8bad640a39895e1cda8b0f24b` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:32.317857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5badbf14-c30d-4dc1-adc0-cff47464a9c0 None admin] RBAC: Authorizing `identity:get_group(group_id=eec63ee8bad640a39895e1cda8b0f24b)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:32.319318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5badbf14-c30d-4dc1-adc0-cff47464a9c0 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:32.321172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 228/449] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 15:10:32 2022] GET /identity/v3/groups/eec63ee8bad640a39895e1cda8b0f24b => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:32.326863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5badbf14-c30d-4dc1-adc0-cff47464a9c0 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:32.327980 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59372e68-0c4e-4e18-b9c5-422166331f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:32.328608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59372e68-0c4e-4e18-b9c5-422166331f49 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:32.328886 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59372e68-0c4e-4e18-b9c5-422166331f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:32.329156 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59372e68-0c4e-4e18-b9c5-422166331f49 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:32.330076 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59372e68-0c4e-4e18-b9c5-422166331f49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:32.330401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59372e68-0c4e-4e18-b9c5-422166331f49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:32.331766 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59372e68-0c4e-4e18-b9c5-422166331f49 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:32.340426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 222/450] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:32.346017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59372e68-0c4e-4e18-b9c5-422166331f49 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:32.428465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5badbf14-c30d-4dc1-adc0-cff47464a9c0 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:32.454978 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59372e68-0c4e-4e18-b9c5-422166331f49 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:32.457021 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:32.467818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1652aa-c788-4c23-b89f-5b8962eda80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:32.468695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1652aa-c788-4c23-b89f-5b8962eda80e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:32.469049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1652aa-c788-4c23-b89f-5b8962eda80e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:32.469442 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1652aa-c788-4c23-b89f-5b8962eda80e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:32.470760 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1652aa-c788-4c23-b89f-5b8962eda80e None admin] RBAC: Adding query filter params (group.id=eec63ee8bad640a39895e1cda8b0f24b, role.id=7ae6dccec1244987a5bc7bf5c3991703, scope.project.id=66624eb1d31e484ba73e6737100a3e5c) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:32.471073 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1652aa-c788-4c23-b89f-5b8962eda80e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:32.472551 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1652aa-c788-4c23-b89f-5b8962eda80e None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:32.477847 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:32.478703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 229/451] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 15:10:32 2022] GET /identity/v3/role_assignments?group.id=eec63ee8bad640a39895e1cda8b0f24b&scope.project.id=66624eb1d31e484ba73e6737100a3e5c&role.id=7ae6dccec1244987a5bc7bf5c3991703 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:32.493037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2c3d8-12b9-4f19-a7ea-edef6c081f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:32.493683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2c3d8-12b9-4f19-a7ea-edef6c081f51 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:32.494026 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2c3d8-12b9-4f19-a7ea-edef6c081f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:32.494298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2c3d8-12b9-4f19-a7ea-edef6c081f51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:32.572046 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2c3d8-12b9-4f19-a7ea-edef6c081f51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:32.573444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2c3d8-12b9-4f19-a7ea-edef6c081f51 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:32.679891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 223/452] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:10:32.686083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1652aa-c788-4c23-b89f-5b8962eda80e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:32.793294 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1652aa-c788-4c23-b89f-5b8962eda80e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:32.827645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adff1f4-466a-486d-b63c-d07aec2226cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:32.828235 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adff1f4-466a-486d-b63c-d07aec2226cc None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:32.828504 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adff1f4-466a-486d-b63c-d07aec2226cc None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:32.828772 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adff1f4-466a-486d-b63c-d07aec2226cc None admin] PATH_INFO: `/v3/projects/ef3b9d837c2248a7ae12fa99e632e776` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:32.830003 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adff1f4-466a-486d-b63c-d07aec2226cc None admin] RBAC: Authorizing `identity:get_project(project_id=ef3b9d837c2248a7ae12fa99e632e776)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:32.831466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adff1f4-466a-486d-b63c-d07aec2226cc None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:32.833144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 230/453] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:10:32 2022] GET /identity/v3/projects/ef3b9d837c2248a7ae12fa99e632e776 => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:32.839021 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2c3d8-12b9-4f19-a7ea-edef6c081f51 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:32.860605 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:32.957692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2c3d8-12b9-4f19-a7ea-edef6c081f51 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:32.995429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e352ef-ed7d-494f-ba88-c40d14127b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:32.996058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e352ef-ed7d-494f-ba88-c40d14127b3f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:32.996313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e352ef-ed7d-494f-ba88-c40d14127b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:32.996614 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e352ef-ed7d-494f-ba88-c40d14127b3f None admin] PATH_INFO: `/v3/groups/bcc4adaf726a4c44a47568d4c737a2f8` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:32.997903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e352ef-ed7d-494f-ba88-c40d14127b3f None admin] RBAC: Authorizing `identity:get_group(group_id=bcc4adaf726a4c44a47568d4c737a2f8)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:32.999371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e352ef-ed7d-494f-ba88-c40d14127b3f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:33.000970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 224/454] 10.222.0.48 () {56 vars in 1180 bytes} [Mon Aug 1 15:10:32 2022] GET /identity/v3/groups/bcc4adaf726a4c44a47568d4c737a2f8 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:33.007496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adff1f4-466a-486d-b63c-d07aec2226cc None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:33.067866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:33.121885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adff1f4-466a-486d-b63c-d07aec2226cc None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:33.161688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc68c5-2629-46e4-8276-8a39033961d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:33.162335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc68c5-2629-46e4-8276-8a39033961d3 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:33.162611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc68c5-2629-46e4-8276-8a39033961d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:33.162879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc68c5-2629-46e4-8276-8a39033961d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:33.163689 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc68c5-2629-46e4-8276-8a39033961d3 None admin] RBAC: Adding query filter params (group.id=bcc4adaf726a4c44a47568d4c737a2f8, role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=ef3b9d837c2248a7ae12fa99e632e776) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:33.164002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc68c5-2629-46e4-8276-8a39033961d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:33.165461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc68c5-2629-46e4-8276-8a39033961d3 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:33.171479 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 231/455] 10.222.0.48 () {56 vars in 1376 bytes} [Mon Aug 1 15:10:33 2022] GET /identity/v3/role_assignments?group.id=bcc4adaf726a4c44a47568d4c737a2f8&scope.project.id=ef3b9d837c2248a7ae12fa99e632e776&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:35.711407 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cd10e-8cb3-47eb-8fea-7e9cdfc99b7e None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:35.711965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cd10e-8cb3-47eb-8fea-7e9cdfc99b7e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:35.711965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155cd10e-8cb3-47eb-8fea-7e9cdfc99b7e None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:35.713560 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 225/456] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:35.718658 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15227132-a9b9-47db-b727-8c3d6c9db00f None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:35.719060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15227132-a9b9-47db-b727-8c3d6c9db00f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:35.719242 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15227132-a9b9-47db-b727-8c3d6c9db00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:35.746808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-15227132-a9b9-47db-b727-8c3d6c9db00f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:35.748088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-15227132-a9b9-47db-b727-8c3d6c9db00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:35.832738 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 232/457] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:35.858867 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cd10e-8cb3-47eb-8fea-7e9cdfc99b7e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:35.913813 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab4741-b59e-4a93-bbd1-eea50d985d8b None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:35.914148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab4741-b59e-4a93-bbd1-eea50d985d8b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:35.914383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab4741-b59e-4a93-bbd1-eea50d985d8b None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:35.915810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 233/458] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:35.922587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645ab8d-14cc-4390-8166-723616623f68 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:35.922587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645ab8d-14cc-4390-8166-723616623f68 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:35.922587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645ab8d-14cc-4390-8166-723616623f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:35.947716 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-e645ab8d-14cc-4390-8166-723616623f68 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:35.948930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e645ab8d-14cc-4390-8166-723616623f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:36.020433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:36.029003 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:36.045051 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155cd10e-8cb3-47eb-8fea-7e9cdfc99b7e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:36.073986 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 234/459] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:10:36.089887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecaf22-460c-477f-999e-2a90a4cf9248 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:36.090172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecaf22-460c-477f-999e-2a90a4cf9248 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:36.090536 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecaf22-460c-477f-999e-2a90a4cf9248 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:36.092289 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 235/460] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:10:36.097233 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24567f77-d738-4523-ae6a-637545c11aee None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:36.097646 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24567f77-d738-4523-ae6a-637545c11aee None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:36.098119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24567f77-d738-4523-ae6a-637545c11aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:36.099136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e220cf-b5a5-4c02-a4e6-a823103fdb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:36.099796 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e220cf-b5a5-4c02-a4e6-a823103fdb45 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:36.100069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e220cf-b5a5-4c02-a4e6-a823103fdb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:36.100338 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e220cf-b5a5-4c02-a4e6-a823103fdb45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:36.111424 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e220cf-b5a5-4c02-a4e6-a823103fdb45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:36.113098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e220cf-b5a5-4c02-a4e6-a823103fdb45 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:36.119733 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-24567f77-d738-4523-ae6a-637545c11aee None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:36.121212 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-24567f77-d738-4523-ae6a-637545c11aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:36.130340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-75e220cf-b5a5-4c02-a4e6-a823103fdb45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:36.130340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:36.130340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:36.130340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:36.130340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:36.130340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:36.130340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:36.130340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:36.130340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:36.130340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:36.130340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:36.130340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:36.130340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:36.130340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:36.130340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:36.131494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:36.137243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:36.137243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:36.137243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:36.137243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:36.137243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 226/461] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:36.137243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e220cf-b5a5-4c02-a4e6-a823103fdb45 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:36.213374 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 236/462] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:36.214549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24567f77-d738-4523-ae6a-637545c11aee None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:36.263582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e220cf-b5a5-4c02-a4e6-a823103fdb45 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:36.299892 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd038aa-bd76-4823-b96a-40f97859750f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:36.300863 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd038aa-bd76-4823-b96a-40f97859750f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:36.301348 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd038aa-bd76-4823-b96a-40f97859750f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:36.301857 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd038aa-bd76-4823-b96a-40f97859750f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:36.311725 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:36.314244 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd038aa-bd76-4823-b96a-40f97859750f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:36.316137 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd038aa-bd76-4823-b96a-40f97859750f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:36.332153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-7dd038aa-bd76-4823-b96a-40f97859750f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:36.332153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:36.332153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:36.332153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:36.332153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:36.332153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:36.332153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:36.332153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:36.332153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:36.332153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:36.332153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:36.332153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:36.332153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:36.332153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:36.332153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:36.333045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:36.333965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:36.333965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:36.333965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:36.333965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:36.336195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 227/463] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:36.337298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd038aa-bd76-4823-b96a-40f97859750f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:36.344375 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24567f77-d738-4523-ae6a-637545c11aee None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:36.359889 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:36.384581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9449c-6417-4413-bdb6-31c4db08b75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:36.385211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9449c-6417-4413-bdb6-31c4db08b75f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:36.385527 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9449c-6417-4413-bdb6-31c4db08b75f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:36.385813 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9449c-6417-4413-bdb6-31c4db08b75f None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:36.386718 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9449c-6417-4413-bdb6-31c4db08b75f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:36.387177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9449c-6417-4413-bdb6-31c4db08b75f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:36.388784 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9449c-6417-4413-bdb6-31c4db08b75f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:36.401649 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 237/464] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:36.402794 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9449c-6417-4413-bdb6-31c4db08b75f None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:36.449361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd038aa-bd76-4823-b96a-40f97859750f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:36.488999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7b90d-ce7a-4c39-bc48-abb9135ff729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:36.489658 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7b90d-ce7a-4c39-bc48-abb9135ff729 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:36.489945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7b90d-ce7a-4c39-bc48-abb9135ff729 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:36.490249 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7b90d-ce7a-4c39-bc48-abb9135ff729 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:36.502864 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7b90d-ce7a-4c39-bc48-abb9135ff729 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:36.504761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7b90d-ce7a-4c39-bc48-abb9135ff729 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:36.519127 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9449c-6417-4413-bdb6-31c4db08b75f None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:36.522673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-29f7b90d-ce7a-4c39-bc48-abb9135ff729 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:36.522673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:36.522673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:36.522673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:36.522673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:36.522673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:36.522673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:36.522673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:36.522673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:36.522673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:36.522673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:36.522673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:36.522673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:36.522673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:36.522673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:36.523527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:36.524378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:36.524378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:36.524378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:36.524378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:36.524378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 228/465] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:36.524636 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7b90d-ce7a-4c39-bc48-abb9135ff729 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:36.556041 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f5138-9072-47c0-9a02-89307b28545d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:36.557045 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f5138-9072-47c0-9a02-89307b28545d None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:36.557621 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f5138-9072-47c0-9a02-89307b28545d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:36.558203 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f5138-9072-47c0-9a02-89307b28545d None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:36.559100 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f5138-9072-47c0-9a02-89307b28545d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:36.559646 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f5138-9072-47c0-9a02-89307b28545d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:36.561325 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f5138-9072-47c0-9a02-89307b28545d None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:36.572961 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:36.580167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 238/466] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:36.581803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab4487-748b-4379-b503-165185c7ea50 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:36.582304 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab4487-748b-4379-b503-165185c7ea50 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:36.582655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab4487-748b-4379-b503-165185c7ea50 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:36.585829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 239/467] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:36.585829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab4487-748b-4379-b503-165185c7ea50 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:36.619793 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:36.634106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7b90d-ce7a-4c39-bc48-abb9135ff729 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:36.668515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50deed-159e-486f-8a2f-59f1d621ef2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:36.669134 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50deed-159e-486f-8a2f-59f1d621ef2a None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:36.669411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50deed-159e-486f-8a2f-59f1d621ef2a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:36.669679 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50deed-159e-486f-8a2f-59f1d621ef2a None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:36.670406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50deed-159e-486f-8a2f-59f1d621ef2a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:36.671845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50deed-159e-486f-8a2f-59f1d621ef2a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:36.705805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 229/468] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:10:36 2022] POST /identity/v3/users => generated 246 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:10:36.705805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50deed-159e-486f-8a2f-59f1d621ef2a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:36.713326 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab4487-748b-4379-b503-165185c7ea50 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:36.769211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c6b79-2925-494d-81b4-57dd0bcebd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:36.769981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c6b79-2925-494d-81b4-57dd0bcebd7b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:36.770609 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c6b79-2925-494d-81b4-57dd0bcebd7b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:36.770952 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c6b79-2925-494d-81b4-57dd0bcebd7b None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:36.772422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c6b79-2925-494d-81b4-57dd0bcebd7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:36.773211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c6b79-2925-494d-81b4-57dd0bcebd7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:36.775181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c6b79-2925-494d-81b4-57dd0bcebd7b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:36.793259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 240/469] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:36.795075 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262bb06-09b5-44a0-9ea2-a523189f914f None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:36.795524 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262bb06-09b5-44a0-9ea2-a523189f914f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:36.795958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262bb06-09b5-44a0-9ea2-a523189f914f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:36.820932 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-8262bb06-09b5-44a0-9ea2-a523189f914f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:36.822780 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8262bb06-09b5-44a0-9ea2-a523189f914f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:36.838525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:36.847262 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50deed-159e-486f-8a2f-59f1d621ef2a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:36.890393 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abdf1f-b60b-4ea1-af81-981d78ee8dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:36.891035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abdf1f-b60b-4ea1-af81-981d78ee8dd1 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:36.891317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abdf1f-b60b-4ea1-af81-981d78ee8dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:36.891591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abdf1f-b60b-4ea1-af81-981d78ee8dd1 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:36.892318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39abdf1f-b60b-4ea1-af81-981d78ee8dd1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:36.893836 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39abdf1f-b60b-4ea1-af81-981d78ee8dd1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:36.912834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 241/470] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:36.913579 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262bb06-09b5-44a0-9ea2-a523189f914f None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:36.923406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 230/471] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:10:36 2022] POST /identity/v3/users => generated 248 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:10:36.924702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b72473-2fd2-4be5-9a4e-77bbed6f7ac4 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:36.924934 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b72473-2fd2-4be5-9a4e-77bbed6f7ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:36.925389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b72473-2fd2-4be5-9a4e-77bbed6f7ac4 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:36.927192 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 231/472] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:36.933290 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087eb94-81aa-4861-8ac9-931741edcdd9 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:36.933948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087eb94-81aa-4861-8ac9-931741edcdd9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:36.934374 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087eb94-81aa-4861-8ac9-931741edcdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:36.951937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:36.956063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4087eb94-81aa-4861-8ac9-931741edcdd9 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:36.957542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4087eb94-81aa-4861-8ac9-931741edcdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:37.048917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 232/473] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:10:37.048917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087eb94-81aa-4861-8ac9-931741edcdd9 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:37.053516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262bb06-09b5-44a0-9ea2-a523189f914f None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:37.108898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8f006-3457-4a11-ab8c-d38b2f1e021d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:37.109454 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8f006-3457-4a11-ab8c-d38b2f1e021d None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:37.109731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8f006-3457-4a11-ab8c-d38b2f1e021d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:37.110032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8f006-3457-4a11-ab8c-d38b2f1e021d None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:37.110752 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8f006-3457-4a11-ab8c-d38b2f1e021d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:37.112251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8f006-3457-4a11-ab8c-d38b2f1e021d None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:37.142805 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 242/474] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:10:36 2022] POST /identity/v3/users => generated 248 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:10:37.143858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8f006-3457-4a11-ab8c-d38b2f1e021d None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:37.175355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:37.198612 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087eb94-81aa-4861-8ac9-931741edcdd9 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:37.250107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:37.258587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628f5ec-4291-45d5-b2c2-280c2ad8f487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:37.259410 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f5ec-4291-45d5-b2c2-280c2ad8f487 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:37.259588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f5ec-4291-45d5-b2c2-280c2ad8f487 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:37.259863 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628f5ec-4291-45d5-b2c2-280c2ad8f487 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:37.278330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628f5ec-4291-45d5-b2c2-280c2ad8f487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:37.279869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628f5ec-4291-45d5-b2c2-280c2ad8f487 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:37.289264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8f006-3457-4a11-ab8c-d38b2f1e021d None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:37.296082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-6628f5ec-4291-45d5-b2c2-280c2ad8f487 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:37.296082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:37.296082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:37.296082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:37.296082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:37.296082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:37.296082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:37.296082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:37.296082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:37.296082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:37.296082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:37.296082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:37.296082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:37.296082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:37.296082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:37.297019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:37.297914 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:37.297914 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:37.297914 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:37.297914 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:37.301278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 233/475] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:37.305761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628f5ec-4291-45d5-b2c2-280c2ad8f487 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:37.355187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd64a19-f48f-4b68-adfa-40af026f3a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:37.359605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd64a19-f48f-4b68-adfa-40af026f3a28 None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:37.360141 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd64a19-f48f-4b68-adfa-40af026f3a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:37.360616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd64a19-f48f-4b68-adfa-40af026f3a28 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:37.361624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd64a19-f48f-4b68-adfa-40af026f3a28 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:37.363671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd64a19-f48f-4b68-adfa-40af026f3a28 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:37.378036 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.sql.core [None req-6dd64a19-f48f-4b68-adfa-40af026f3a28 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 15:10:37.378036 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 15:10:37.378036 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [parameters: {'id': '58a6816414b640f2a8fe543b658380e4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 15:10:37.378036 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80292) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 15:10:37.397342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-6dd64a19-f48f-4b68-adfa-40af026f3a28 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 15:10:37.397342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:37.397342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:10:37.397342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:10:37.397342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:10:37.397342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:10:37.397342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:10:37.397342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:10:37.397342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:10:37.397342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:10:37.397342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:10:37.397342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:10:37.397342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:10:37.397342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application result.read() Aug 01 15:10:37.397342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return ref.to_dict() Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:10:37.398307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application yield resource Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise self.value Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application session.commit() Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 15:10:37.399253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self.session.flush() Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application compat.raise_( Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 15:10:37.400126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 15:10:37.401025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 15:10:37.401025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 15:10:37.401025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 15:10:37.401025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 15:10:37.401025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 15:10:37.401025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 15:10:37.401025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 15:10:37.401025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 15:10:37.401025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 15:10:37.401025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 15:10:37.401025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 15:10:37.401025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 15:10:37.401025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 15:10:37.401025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 15:10:37.401025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:10:37.401025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception Aug 01 15:10:37.401877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:10:37.401877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:10:37.401877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:10:37.401877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:10:37.401877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:10:37.401877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:10:37.401877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:10:37.401877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:10:37.401877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:10:37.401877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:10:37.401877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:10:37.401877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application result.read() Aug 01 15:10:37.401877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:10:37.401877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:10:37.401877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:10:37.401877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:10:37.402797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:10:37.402797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:10:37.402797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:10:37.402797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:10:37.402797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 15:10:37.402797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 15:10:37.402797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [parameters: {'id': '58a6816414b640f2a8fe543b658380e4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 15:10:37.402797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 15:10:37.402797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:37.402797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:10:37.402797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:37.402797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:37.402797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:37.402797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:37.402797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:37.402797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:37.403860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:37.403860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:37.403860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:37.403860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:37.403860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:37.403860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:37.403860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 01 15:10:37.403860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 01 15:10:37.403860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:37.403860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:37.403860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 01 15:10:37.403860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 01 15:10:37.403860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 15:10:37.403860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 15:10:37.403860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 15:10:37.403860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:37.406718 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 243/476] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 1 15:10:37 2022] POST /identity/v3/roles => generated 144 bytes in 263 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:37.412269 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd64a19-f48f-4b68-adfa-40af026f3a28 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:37.441449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628f5ec-4291-45d5-b2c2-280c2ad8f487 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:37.480383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcdae0-615b-41c8-b314-2e879a871d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:37.481123 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcdae0-615b-41c8-b314-2e879a871d76 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:37.481531 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcdae0-615b-41c8-b314-2e879a871d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:37.481825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcdae0-615b-41c8-b314-2e879a871d76 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:37.482537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcdae0-615b-41c8-b314-2e879a871d76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:37.483202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcdae0-615b-41c8-b314-2e879a871d76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:37.484600 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcdae0-615b-41c8-b314-2e879a871d76 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:37.495705 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:37.509895 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 234/477] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:37.510861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcdae0-615b-41c8-b314-2e879a871d76 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:37.528949 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd64a19-f48f-4b68-adfa-40af026f3a28 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:37.567775 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb1f5d-b928-4602-b2b2-b98c7871cb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:37.568940 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb1f5d-b928-4602-b2b2-b98c7871cb88 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:37.569403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb1f5d-b928-4602-b2b2-b98c7871cb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:37.569883 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb1f5d-b928-4602-b2b2-b98c7871cb88 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:37.583053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb1f5d-b928-4602-b2b2-b98c7871cb88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:37.590070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-cfbb1f5d-b928-4602-b2b2-b98c7871cb88 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:37.590070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:37.590070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:37.590070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:37.590070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:37.590070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:37.590070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:37.590070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:37.590070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:37.590070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:37.590070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:37.590070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:37.590070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:37.590070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:37.590070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:37.590070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:37.591002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:37.591002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:37.591002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:37.591002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:37.591002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:37.591002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:37.591002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:37.591002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:37.591002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:37.591002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:37.591002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:37.591002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:37.591002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:37.591002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:37.591002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:37.591002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:37.591837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:37.591837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:37.591837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:37.591837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:37.591837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:37.591837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:37.591837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:37.591837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:37.591837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:37.591837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:37.591837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:10:37.591837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:37.594679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 244/478] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:10:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:37.600397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb1f5d-b928-4602-b2b2-b98c7871cb88 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:37.624823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcdae0-615b-41c8-b314-2e879a871d76 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:37.635679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:37.660991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca1e30-ffae-45b4-93fd-3badc87d8e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:37.661726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca1e30-ffae-45b4-93fd-3badc87d8e27 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:37.662099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca1e30-ffae-45b4-93fd-3badc87d8e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:37.662436 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca1e30-ffae-45b4-93fd-3badc87d8e27 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:37.663194 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca1e30-ffae-45b4-93fd-3badc87d8e27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:37.664762 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca1e30-ffae-45b4-93fd-3badc87d8e27 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:37.694294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 235/479] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:10:37 2022] POST /identity/v3/users => generated 249 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:10:37.739735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb1f5d-b928-4602-b2b2-b98c7871cb88 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:37.792260 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b82bc-37fe-4d6f-a90b-32dcb15a6742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:37.792933 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b82bc-37fe-4d6f-a90b-32dcb15a6742 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:37.793165 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b82bc-37fe-4d6f-a90b-32dcb15a6742 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:37.793442 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b82bc-37fe-4d6f-a90b-32dcb15a6742 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:37.794221 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b82bc-37fe-4d6f-a90b-32dcb15a6742 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:37.794533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b82bc-37fe-4d6f-a90b-32dcb15a6742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:37.795984 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b82bc-37fe-4d6f-a90b-32dcb15a6742 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:37.805949 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 245/480] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:38.905297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e569ea-8f84-4763-a21b-9405a6cd528b None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:38.905848 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e569ea-8f84-4763-a21b-9405a6cd528b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:38.905928 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e569ea-8f84-4763-a21b-9405a6cd528b None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:38.907561 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 236/481] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:38.912839 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a04b6-7258-4b29-9eed-072f5088c57a None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:38.913243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a04b6-7258-4b29-9eed-072f5088c57a None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:38.913408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a04b6-7258-4b29-9eed-072f5088c57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:38.932742 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-fe2a04b6-7258-4b29-9eed-072f5088c57a None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:38.934163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-fe2a04b6-7258-4b29-9eed-072f5088c57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:38.996897 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:39.006484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 246/482] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:39.029874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e569ea-8f84-4763-a21b-9405a6cd528b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:39.109070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:39.161060 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e569ea-8f84-4763-a21b-9405a6cd528b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:39.195211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc97249-7494-44c3-83ac-264fc15f3749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:39.195867 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc97249-7494-44c3-83ac-264fc15f3749 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:39.196148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc97249-7494-44c3-83ac-264fc15f3749 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:39.196420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc97249-7494-44c3-83ac-264fc15f3749 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:39.208386 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc97249-7494-44c3-83ac-264fc15f3749 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:39.214086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-9bc97249-7494-44c3-83ac-264fc15f3749 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:39.214086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:39.214086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:39.214086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:39.214086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:39.214086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:39.214086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:39.214086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:39.214086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:39.214086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:39.214086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:39.214086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:39.214086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:39.214086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:39.214086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:39.214086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:39.215205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:39.215205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:39.215205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:39.215205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:39.215205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:39.215205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:39.215205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:39.215205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:39.215205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:39.215205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:39.215205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:39.215205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:39.215205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:39.215205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:39.215205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:39.215205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:39.216173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:39.216173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:39.216173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:39.216173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:39.216173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:39.216173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:39.216173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:39.216173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:39.216173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:39.216173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:39.216173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:39.216173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:39.216173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 237/483] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:39.221922 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a04b6-7258-4b29-9eed-072f5088c57a None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:39.352801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a04b6-7258-4b29-9eed-072f5088c57a None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:39.397324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8d13b-5471-4295-8911-e07b566e20d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:39.398228 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d13b-5471-4295-8911-e07b566e20d9 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:39.398644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d13b-5471-4295-8911-e07b566e20d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:39.399083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d13b-5471-4295-8911-e07b566e20d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:39.399960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8d13b-5471-4295-8911-e07b566e20d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:39.400403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8d13b-5471-4295-8911-e07b566e20d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:39.401961 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8d13b-5471-4295-8911-e07b566e20d9 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:39.411599 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 247/484] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:10:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:39.417026 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc97249-7494-44c3-83ac-264fc15f3749 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:39.533068 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3ec2d-343a-4266-bdd2-519a1477ad4c None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:39.533339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3ec2d-343a-4266-bdd2-519a1477ad4c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:39.533613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3ec2d-343a-4266-bdd2-519a1477ad4c None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:39.537921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 248/485] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:10:39.540807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7394e8-2d78-46f2-bcce-40702739dc3d None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:39.544092 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7394e8-2d78-46f2-bcce-40702739dc3d None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:39.544092 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7394e8-2d78-46f2-bcce-40702739dc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:39.583086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc97249-7494-44c3-83ac-264fc15f3749 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:39.584340 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-bd7394e8-2d78-46f2-bcce-40702739dc3d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:39.585736 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-bd7394e8-2d78-46f2-bcce-40702739dc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:39.614687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:39.643461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eff214-2275-4ec5-ad85-8d6fb5594510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:39.644423 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eff214-2275-4ec5-ad85-8d6fb5594510 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:39.644873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eff214-2275-4ec5-ad85-8d6fb5594510 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:39.645313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eff214-2275-4ec5-ad85-8d6fb5594510 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:39.666272 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eff214-2275-4ec5-ad85-8d6fb5594510 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:39.668708 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eff214-2275-4ec5-ad85-8d6fb5594510 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:39.693142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e9eff214-2275-4ec5-ad85-8d6fb5594510 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:39.693142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:39.693142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:39.693142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:39.693142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:39.693142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:39.693142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:39.693142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:39.693142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:39.693142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:39.693142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:39.693142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:39.693142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:39.693142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:39.693142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:39.694166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:39.695380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:39.695380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:39.695380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:39.695380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:39.701928 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 238/486] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:39.704134 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eff214-2275-4ec5-ad85-8d6fb5594510 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:39.710127 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:39.742004 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 249/487] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:10:39.777079 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7394e8-2d78-46f2-bcce-40702739dc3d None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:39.821710 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eff214-2275-4ec5-ad85-8d6fb5594510 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:39.871433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c2f56-beb0-43a6-83a9-37d2bb343097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:39.872385 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c2f56-beb0-43a6-83a9-37d2bb343097 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:39.872866 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c2f56-beb0-43a6-83a9-37d2bb343097 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:39.873315 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c2f56-beb0-43a6-83a9-37d2bb343097 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:39.874264 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c2f56-beb0-43a6-83a9-37d2bb343097 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:39.874748 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c2f56-beb0-43a6-83a9-37d2bb343097 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:39.876401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c2f56-beb0-43a6-83a9-37d2bb343097 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:39.897486 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 239/488] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:39.902919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c2f56-beb0-43a6-83a9-37d2bb343097 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:39.929935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7394e8-2d78-46f2-bcce-40702739dc3d None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:39.954372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:39.967655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb6572-293f-4737-ac7b-d35ced9a2302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:39.968603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb6572-293f-4737-ac7b-d35ced9a2302 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:39.969057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb6572-293f-4737-ac7b-d35ced9a2302 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:39.969503 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb6572-293f-4737-ac7b-d35ced9a2302 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:39.980647 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb6572-293f-4737-ac7b-d35ced9a2302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:39.986408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-79fb6572-293f-4737-ac7b-d35ced9a2302 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:39.986408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:39.986408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:39.986408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:39.986408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:39.986408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:39.986408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:39.986408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:39.986408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:39.986408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:39.986408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:39.986408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:39.986408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:39.986408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:39.986408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:39.986408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:39.987278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:39.987278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:39.987278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:39.987278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:39.987278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:39.987278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:39.987278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:39.987278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:39.987278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:39.987278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:39.987278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:39.987278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:39.987278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:39.987278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:39.987278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:39.987278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:39.988158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:39.988158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:39.988158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:39.988158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:39.988158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:39.988158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:39.988158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:39.988158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:39.988158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:39.988158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:39.988158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:39.988158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:39.990883 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 250/489] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:39.996861 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb6572-293f-4737-ac7b-d35ced9a2302 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:40.046296 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:40.117971 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c2f56-beb0-43a6-83a9-37d2bb343097 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:40.125342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb6572-293f-4737-ac7b-d35ced9a2302 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:40.160450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5a009-1f4b-4c61-a758-2c208b9d161e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:40.161632 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5a009-1f4b-4c61-a758-2c208b9d161e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:40.162512 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5a009-1f4b-4c61-a758-2c208b9d161e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:40.163257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5a009-1f4b-4c61-a758-2c208b9d161e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:40.167655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e50a95-fa14-4803-a2f9-4ba67b04fd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:40.167828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e50a95-fa14-4803-a2f9-4ba67b04fd95 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:40.168064 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e50a95-fa14-4803-a2f9-4ba67b04fd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:40.168248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e50a95-fa14-4803-a2f9-4ba67b04fd95 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:40.168993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e50a95-fa14-4803-a2f9-4ba67b04fd95 None admin] RBAC: Adding query filter params (name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:40.169299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e50a95-fa14-4803-a2f9-4ba67b04fd95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:40.170735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e50a95-fa14-4803-a2f9-4ba67b04fd95 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:40.178626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5a009-1f4b-4c61-a758-2c208b9d161e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:40.182266 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5a009-1f4b-4c61-a758-2c208b9d161e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:40.187490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 251/490] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:10:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:40.187490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a504657-9ee0-43fd-bf75-5ef5a8c496d2 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:40.187490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a504657-9ee0-43fd-bf75-5ef5a8c496d2 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:40.187490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a504657-9ee0-43fd-bf75-5ef5a8c496d2 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:40.190759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 252/491] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:40.192247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a504657-9ee0-43fd-bf75-5ef5a8c496d2 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:40.206510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-40c5a009-1f4b-4c61-a758-2c208b9d161e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:10:40.206510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:40.206510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:40.206510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:40.206510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:40.206510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:40.206510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:40.206510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:40.206510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:40.206510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:40.206510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:40.206510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:40.206510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:10:40.206510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:10:40.206510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:10:40.206510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:40.207460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:40.208366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:40.208366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:40.208366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:10:40.208366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:10:40.208366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:10:40.208366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:10:40.208366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:10:40.208366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:10:40.208366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:10:40.208366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:40.211398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 240/492] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 15:10:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:40.213363 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c312bb3-fa5e-458a-a4a3-095f1c772dcf None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:40.213968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c312bb3-fa5e-458a-a4a3-095f1c772dcf None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:40.214437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c312bb3-fa5e-458a-a4a3-095f1c772dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:40.239465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3c312bb3-fa5e-458a-a4a3-095f1c772dcf None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:40.241473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3c312bb3-fa5e-458a-a4a3-095f1c772dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:40.320508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a504657-9ee0-43fd-bf75-5ef5a8c496d2 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:40.331107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 241/493] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:40.337255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c312bb3-fa5e-458a-a4a3-095f1c772dcf None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:40.339678 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:40.371637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7945a-e1b5-49c0-827c-773ba5ec4172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:40.373508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7945a-e1b5-49c0-827c-773ba5ec4172 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:40.375415 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7945a-e1b5-49c0-827c-773ba5ec4172 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:40.375901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7945a-e1b5-49c0-827c-773ba5ec4172 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:40.377154 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:40.390590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7945a-e1b5-49c0-827c-773ba5ec4172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:40.392524 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7945a-e1b5-49c0-827c-773ba5ec4172 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:40.410023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-dfe7945a-e1b5-49c0-827c-773ba5ec4172 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:40.410023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:40.410023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:40.410023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:40.410023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:40.410023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:40.410023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:40.410023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:40.410023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:40.410023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:40.410023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:40.410023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:40.410023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:40.410023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:40.410023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:40.410956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:40.412555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:40.412555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:40.412555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:40.412555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:40.415249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 253/494] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:40.420874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7945a-e1b5-49c0-827c-773ba5ec4172 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:40.475015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c312bb3-fa5e-458a-a4a3-095f1c772dcf None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:40.521238 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af256c29-6948-43c7-9b15-5d0915a35586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:40.521929 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af256c29-6948-43c7-9b15-5d0915a35586 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:40.522202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af256c29-6948-43c7-9b15-5d0915a35586 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:40.522474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af256c29-6948-43c7-9b15-5d0915a35586 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:40.523286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af256c29-6948-43c7-9b15-5d0915a35586 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:40.523589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af256c29-6948-43c7-9b15-5d0915a35586 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:40.525105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af256c29-6948-43c7-9b15-5d0915a35586 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:40.533673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7945a-e1b5-49c0-827c-773ba5ec4172 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:40.565500 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 242/495] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 15:10:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:40.566388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af256c29-6948-43c7-9b15-5d0915a35586 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:40.569214 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205a2b9-7dc1-4087-9670-7385efa111cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:40.570533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205a2b9-7dc1-4087-9670-7385efa111cc None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:40.570533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205a2b9-7dc1-4087-9670-7385efa111cc None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:40.570533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205a2b9-7dc1-4087-9670-7385efa111cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:40.581259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3205a2b9-7dc1-4087-9670-7385efa111cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:40.586774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-3205a2b9-7dc1-4087-9670-7385efa111cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:40.586774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:40.586774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:40.586774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:40.586774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:40.586774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:40.586774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:40.586774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:40.586774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:40.586774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:40.586774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:40.586774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:40.586774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:40.586774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:40.586774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:40.586774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:40.587789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:40.587789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:40.587789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:40.587789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:40.587789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:40.587789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:40.587789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:40.587789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:40.587789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:40.587789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:40.587789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:40.587789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:40.587789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:40.587789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:40.587789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:40.587789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:40.588594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:40.588594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:40.588594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:40.588594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:40.588594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:40.588594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:40.588594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:40.588594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:40.588594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:40.588594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:40.588594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:40.588594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:40.588594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 254/496] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:40.589276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601b9bb-720c-44bf-9c85-687b8f425321 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:40.589344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601b9bb-720c-44bf-9c85-687b8f425321 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:40.589894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601b9bb-720c-44bf-9c85-687b8f425321 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:40.591114 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 255/497] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:40.592001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce48f0c-7855-4645-ba20-97064bcf38d7 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:40.592283 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce48f0c-7855-4645-ba20-97064bcf38d7 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:40.592543 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce48f0c-7855-4645-ba20-97064bcf38d7 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:40.593726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 256/498] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:40.594432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce48f0c-7855-4645-ba20-97064bcf38d7 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:40.626418 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:40.656438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:40.684877 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af256c29-6948-43c7-9b15-5d0915a35586 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:40.712509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce48f0c-7855-4645-ba20-97064bcf38d7 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:40.720685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a75a24-28bd-4867-81a2-c8e3f57bdc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:40.721333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a75a24-28bd-4867-81a2-c8e3f57bdc29 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:40.721612 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a75a24-28bd-4867-81a2-c8e3f57bdc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:40.721884 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a75a24-28bd-4867-81a2-c8e3f57bdc29 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:40.722584 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a75a24-28bd-4867-81a2-c8e3f57bdc29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:40.722888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a75a24-28bd-4867-81a2-c8e3f57bdc29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:40.725235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a75a24-28bd-4867-81a2-c8e3f57bdc29 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:40.739393 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 243/499] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:40.740261 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a75a24-28bd-4867-81a2-c8e3f57bdc29 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:40.751857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee5c2e-7d80-4d3e-8912-3cf048dc0a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:40.752484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee5c2e-7d80-4d3e-8912-3cf048dc0a2f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:40.752766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee5c2e-7d80-4d3e-8912-3cf048dc0a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:40.753033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee5c2e-7d80-4d3e-8912-3cf048dc0a2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:40.825965 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee5c2e-7d80-4d3e-8912-3cf048dc0a2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:40.827636 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee5c2e-7d80-4d3e-8912-3cf048dc0a2f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:40.842892 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a75a24-28bd-4867-81a2-c8e3f57bdc29 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:40.848957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:40.880488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2ffc1-82d8-41f5-b762-1ba8e8390db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:40.881235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2ffc1-82d8-41f5-b762-1ba8e8390db4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:40.881507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2ffc1-82d8-41f5-b762-1ba8e8390db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:40.881790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2ffc1-82d8-41f5-b762-1ba8e8390db4 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:40.882636 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2ffc1-82d8-41f5-b762-1ba8e8390db4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:40.882934 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2ffc1-82d8-41f5-b762-1ba8e8390db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:40.884436 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2ffc1-82d8-41f5-b762-1ba8e8390db4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:40.885986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:40.893773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 244/500] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:10:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:40.895017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431cb3f-8256-4153-857c-8584d88b4bcb None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:40.895303 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431cb3f-8256-4153-857c-8584d88b4bcb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:40.895580 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2431cb3f-8256-4153-857c-8584d88b4bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:40.913808 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2431cb3f-8256-4153-857c-8584d88b4bcb None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:40.915058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2431cb3f-8256-4153-857c-8584d88b4bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:40.944166 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 257/501] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:40.945452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb49f2-bc3b-4857-b89e-91db12d00766 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:40.945866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb49f2-bc3b-4857-b89e-91db12d00766 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:40.945976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb49f2-bc3b-4857-b89e-91db12d00766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:40.965850 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-40fb49f2-bc3b-4857-b89e-91db12d00766 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:40.967120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-40fb49f2-bc3b-4857-b89e-91db12d00766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:40.984830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 245/502] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:40.985882 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431cb3f-8256-4153-857c-8584d88b4bcb None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:41.042025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 258/503] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:41.042883 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb49f2-bc3b-4857-b89e-91db12d00766 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:41.087688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2431cb3f-8256-4153-857c-8584d88b4bcb None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:41.108923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:41.117599 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:41.125589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8299bdd-a121-4049-a9a2-830575a4b0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:41.126295 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8299bdd-a121-4049-a9a2-830575a4b0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:41.126573 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8299bdd-a121-4049-a9a2-830575a4b0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:41.126843 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8299bdd-a121-4049-a9a2-830575a4b0e4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:41.134011 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8299bdd-a121-4049-a9a2-830575a4b0e4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:41.135473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8299bdd-a121-4049-a9a2-830575a4b0e4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:41.149235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-a8299bdd-a121-4049-a9a2-830575a4b0e4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:10:41.149235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:41.149235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:41.149235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:41.149235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:41.149235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:41.149235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:41.149235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:41.149235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:41.149235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:41.149235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:41.149235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:41.149235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:10:41.149235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:10:41.149235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:10:41.149235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:41.150279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:41.151341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:41.151341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:41.151341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:10:41.151341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:10:41.151341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:10:41.151341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:10:41.151341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:10:41.151341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:10:41.151341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:10:41.151341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:41.151341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 246/504] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:41.151341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8299bdd-a121-4049-a9a2-830575a4b0e4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:41.166596 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb49f2-bc3b-4857-b89e-91db12d00766 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:41.202680 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ecc7e-ac59-4a90-b50c-039bd6cc6761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:41.203585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ecc7e-ac59-4a90-b50c-039bd6cc6761 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:41.203863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ecc7e-ac59-4a90-b50c-039bd6cc6761 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:41.204133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ecc7e-ac59-4a90-b50c-039bd6cc6761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:41.215696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ecc7e-ac59-4a90-b50c-039bd6cc6761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:41.217282 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ecc7e-ac59-4a90-b50c-039bd6cc6761 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:41.234181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-5b0ecc7e-ac59-4a90-b50c-039bd6cc6761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:41.234181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:41.234181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:41.234181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:41.234181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:41.234181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:41.234181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:41.234181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:41.234181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:41.234181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:41.234181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:41.234181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:41.234181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:41.234181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:41.234181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:41.235497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:41.236778 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:41.236778 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:41.236778 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:41.236778 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:41.236778 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 259/505] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:41.236778 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ecc7e-ac59-4a90-b50c-039bd6cc6761 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:41.266073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8299bdd-a121-4049-a9a2-830575a4b0e4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:41.301752 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8508a-51c3-4026-b52c-7c77dd154d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:41.302535 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8508a-51c3-4026-b52c-7c77dd154d46 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:41.302714 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8508a-51c3-4026-b52c-7c77dd154d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:41.302991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8508a-51c3-4026-b52c-7c77dd154d46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:41.313704 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8508a-51c3-4026-b52c-7c77dd154d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:41.315204 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8508a-51c3-4026-b52c-7c77dd154d46 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:41.329199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-66b8508a-51c3-4026-b52c-7c77dd154d46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:41.329199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:41.329199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:41.329199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:41.329199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:41.329199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:41.329199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:41.329199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:41.329199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:41.329199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:41.329199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:41.329199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:41.329199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:41.329199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:41.329199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:41.330492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:41.331345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:41.331345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:41.331345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:41.331345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:41.331345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 247/506] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:41.331605 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8508a-51c3-4026-b52c-7c77dd154d46 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:41.345251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ecc7e-ac59-4a90-b50c-039bd6cc6761 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:41.346825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:41.373532 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:41.382426 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15decf15-c7d2-4c67-bd5f-933b341166ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:41.383087 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15decf15-c7d2-4c67-bd5f-933b341166ed None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:41.383369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15decf15-c7d2-4c67-bd5f-933b341166ed None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:41.383643 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15decf15-c7d2-4c67-bd5f-933b341166ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:41.394777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15decf15-c7d2-4c67-bd5f-933b341166ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:41.396247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15decf15-c7d2-4c67-bd5f-933b341166ed None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:41.410804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-15decf15-c7d2-4c67-bd5f-933b341166ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:41.410804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:41.410804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:41.410804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:41.410804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:41.410804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:41.410804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:41.410804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:41.410804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:41.410804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:41.410804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:41.410804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:41.410804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:41.410804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:41.410804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:41.411667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:41.412539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:41.412539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:41.412539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:41.412539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:41.412539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 260/507] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:41.412799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15decf15-c7d2-4c67-bd5f-933b341166ed None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:41.436493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8508a-51c3-4026-b52c-7c77dd154d46 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:41.472433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bf84d-a030-4037-bf64-ae9775eba0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:41.473178 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bf84d-a030-4037-bf64-ae9775eba0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:41.473464 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bf84d-a030-4037-bf64-ae9775eba0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:41.473737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bf84d-a030-4037-bf64-ae9775eba0c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:41.484274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bf84d-a030-4037-bf64-ae9775eba0c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:41.489769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-888bf84d-a030-4037-bf64-ae9775eba0c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:41.489769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:41.489769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:41.489769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:41.489769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:41.489769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:41.489769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:41.489769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:41.489769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:41.489769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:41.489769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:41.489769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:41.489769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:41.489769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:41.489769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:41.489769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:41.490875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:41.490875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:41.490875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:41.490875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:41.490875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:41.490875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:41.490875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:41.490875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:41.490875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:41.490875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:41.490875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:41.490875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:41.490875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:41.490875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:41.490875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:41.490875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:41.491946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:41.491946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:41.491946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:41.491946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:41.491946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:41.491946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:41.491946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:41.491946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:41.491946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:41.491946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:41.491946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:41.491946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:41.491946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 248/508] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:41.492701 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bf84d-a030-4037-bf64-ae9775eba0c2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:41.522444 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15decf15-c7d2-4c67-bd5f-933b341166ed None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:41.555482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8eba5-1294-4f01-863b-dabc1c0897c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:41.556107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8eba5-1294-4f01-863b-dabc1c0897c2 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:41.556384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8eba5-1294-4f01-863b-dabc1c0897c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:41.556662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8eba5-1294-4f01-863b-dabc1c0897c2 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:41.557471 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8eba5-1294-4f01-863b-dabc1c0897c2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:41.557801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8eba5-1294-4f01-863b-dabc1c0897c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:41.559344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8eba5-1294-4f01-863b-dabc1c0897c2 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:41.586103 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:41.590146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bf84d-a030-4037-bf64-ae9775eba0c2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:41.596276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 261/509] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 15:10:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:41.597137 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8eba5-1294-4f01-863b-dabc1c0897c2 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:41.611970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:41.628993 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec25b6-790b-4976-a34d-a9ec611b8d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:41.629626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec25b6-790b-4976-a34d-a9ec611b8d08 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:41.629947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec25b6-790b-4976-a34d-a9ec611b8d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:41.630220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec25b6-790b-4976-a34d-a9ec611b8d08 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:41.630922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec25b6-790b-4976-a34d-a9ec611b8d08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:41.631223 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec25b6-790b-4976-a34d-a9ec611b8d08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:41.632621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec25b6-790b-4976-a34d-a9ec611b8d08 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:41.645757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 249/510] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:41.646647 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec25b6-790b-4976-a34d-a9ec611b8d08 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:41.708396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8eba5-1294-4f01-863b-dabc1c0897c2 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:41.746305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63d7c2-b206-4090-866a-d39108a75104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:41.746850 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63d7c2-b206-4090-866a-d39108a75104 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:41.747130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63d7c2-b206-4090-866a-d39108a75104 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:41.747396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63d7c2-b206-4090-866a-d39108a75104 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:41.748096 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63d7c2-b206-4090-866a-d39108a75104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:41.748401 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63d7c2-b206-4090-866a-d39108a75104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:41.750079 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63d7c2-b206-4090-866a-d39108a75104 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:41.754830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec25b6-790b-4976-a34d-a9ec611b8d08 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:41.761934 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 262/511] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:41.762844 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63d7c2-b206-4090-866a-d39108a75104 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:41.799449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf324ae-5f64-4c26-b2c2-52abceb85842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:41.800074 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf324ae-5f64-4c26-b2c2-52abceb85842 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:41.800354 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf324ae-5f64-4c26-b2c2-52abceb85842 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:41.800627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf324ae-5f64-4c26-b2c2-52abceb85842 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:41.801370 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf324ae-5f64-4c26-b2c2-52abceb85842 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:41.801668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf324ae-5f64-4c26-b2c2-52abceb85842 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:41.803136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf324ae-5f64-4c26-b2c2-52abceb85842 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:41.814740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 250/512] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:41.815573 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf324ae-5f64-4c26-b2c2-52abceb85842 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:41.836461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:41.837614 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:41.872334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63d7c2-b206-4090-866a-d39108a75104 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:41.907346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f3635-6abe-4d37-b99f-52eee258d85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:41.908002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f3635-6abe-4d37-b99f-52eee258d85b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:41.908284 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f3635-6abe-4d37-b99f-52eee258d85b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:41.908557 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f3635-6abe-4d37-b99f-52eee258d85b None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:41.909293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f3635-6abe-4d37-b99f-52eee258d85b None admin] RBAC: Adding query filter params (name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:41.909589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f3635-6abe-4d37-b99f-52eee258d85b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:41.911031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f3635-6abe-4d37-b99f-52eee258d85b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:41.918963 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 263/513] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:10:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:41.919772 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f3635-6abe-4d37-b99f-52eee258d85b None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:41.921189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf324ae-5f64-4c26-b2c2-52abceb85842 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:41.959401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36d6f9-102f-40a0-9e0d-2aaf5e2c7978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:41.960019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d6f9-102f-40a0-9e0d-2aaf5e2c7978 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:41.960299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d6f9-102f-40a0-9e0d-2aaf5e2c7978 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:41.960562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36d6f9-102f-40a0-9e0d-2aaf5e2c7978 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:42.022476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f3635-6abe-4d37-b99f-52eee258d85b None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:42.032157 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36d6f9-102f-40a0-9e0d-2aaf5e2c7978 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:42.033640 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da36d6f9-102f-40a0-9e0d-2aaf5e2c7978 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:42.052423 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:42.058223 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490867ea-2eca-456c-b3fe-d31785a18983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:42.058863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490867ea-2eca-456c-b3fe-d31785a18983 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:42.059163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490867ea-2eca-456c-b3fe-d31785a18983 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:42.059419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490867ea-2eca-456c-b3fe-d31785a18983 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:42.061369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:42.067818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490867ea-2eca-456c-b3fe-d31785a18983 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:42.069482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490867ea-2eca-456c-b3fe-d31785a18983 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:42.082291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-490867ea-2eca-456c-b3fe-d31785a18983 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:10:42.082291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:42.082291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:42.082291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:42.082291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:42.082291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:42.082291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:42.082291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:42.082291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:42.082291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:42.082291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:42.082291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:42.082291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:10:42.082291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:10:42.082291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:10:42.082291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:42.083206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:42.084130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:42.084130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:42.084130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:10:42.084130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:10:42.084130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:10:42.084130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:10:42.084130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:10:42.084130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:10:42.084130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:10:42.084130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:42.084130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 264/514] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:42.084860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490867ea-2eca-456c-b3fe-d31785a18983 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:42.141353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 251/515] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:42.142257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36d6f9-102f-40a0-9e0d-2aaf5e2c7978 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:42.190631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490867ea-2eca-456c-b3fe-d31785a18983 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:42.224614 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c642b-fdb3-4785-a644-23105e3a252f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:42.225255 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c642b-fdb3-4785-a644-23105e3a252f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:42.225530 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c642b-fdb3-4785-a644-23105e3a252f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:42.225816 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c642b-fdb3-4785-a644-23105e3a252f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:42.233037 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c642b-fdb3-4785-a644-23105e3a252f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:42.234567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c642b-fdb3-4785-a644-23105e3a252f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:42.244576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36d6f9-102f-40a0-9e0d-2aaf5e2c7978 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:42.246624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-686c642b-fdb3-4785-a644-23105e3a252f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:42.246624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:42.246624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:42.246624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:42.246624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:42.246624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:42.246624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:42.246624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:42.246624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:42.246624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:42.246624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:42.246624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:42.246624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:10:42.246624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:10:42.246624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:42.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:42.248322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:10:42.248322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:10:42.248322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:10:42.248322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:10:42.248322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:10:42.248322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:10:42.248322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:42.248322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:42.248322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 265/516] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:42.248802 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c642b-fdb3-4785-a644-23105e3a252f None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:42.266594 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:42.283096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f7022-01cf-469f-9972-145adda01020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:42.283726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f7022-01cf-469f-9972-145adda01020 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:42.284003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f7022-01cf-469f-9972-145adda01020 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:42.284277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f7022-01cf-469f-9972-145adda01020 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:42.284953 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f7022-01cf-469f-9972-145adda01020 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:42.286448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f7022-01cf-469f-9972-145adda01020 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:42.304352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:42.308577 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 252/517] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:10:42 2022] POST /identity/v3/users => generated 247 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:10:42.309459 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f7022-01cf-469f-9972-145adda01020 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:42.355376 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c642b-fdb3-4785-a644-23105e3a252f None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:42.393510 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcbf5e-1dcb-4ad0-bcc0-766ad3b567bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:42.394211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcbf5e-1dcb-4ad0-bcc0-766ad3b567bd None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:42.394496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcbf5e-1dcb-4ad0-bcc0-766ad3b567bd None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:42.394774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcbf5e-1dcb-4ad0-bcc0-766ad3b567bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:42.406978 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fcbf5e-1dcb-4ad0-bcc0-766ad3b567bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:42.408578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fcbf5e-1dcb-4ad0-bcc0-766ad3b567bd None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:42.424511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-69fcbf5e-1dcb-4ad0-bcc0-766ad3b567bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:42.424511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:42.424511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:42.424511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:42.424511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:42.424511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:42.424511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:42.424511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:42.424511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:42.424511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:42.424511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:42.424511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:42.424511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:42.424511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:42.424511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:42.425660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:42.426553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:42.426553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:42.426553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:42.426553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:42.426553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 266/518] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:42.426553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcbf5e-1dcb-4ad0-bcc0-766ad3b567bd None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:42.443325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f7022-01cf-469f-9972-145adda01020 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:42.483911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21cd26-31ee-4705-9d38-66db462b9889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:42.484549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21cd26-31ee-4705-9d38-66db462b9889 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:42.484810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21cd26-31ee-4705-9d38-66db462b9889 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:42.485106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21cd26-31ee-4705-9d38-66db462b9889 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:42.486075 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21cd26-31ee-4705-9d38-66db462b9889 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:42.486390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21cd26-31ee-4705-9d38-66db462b9889 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:42.487868 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21cd26-31ee-4705-9d38-66db462b9889 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:42.507474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 253/519] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 15:10:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:42.508353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21cd26-31ee-4705-9d38-66db462b9889 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:42.540793 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcbf5e-1dcb-4ad0-bcc0-766ad3b567bd None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:42.547637 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:42.551200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:42.579754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04d8fb-8adf-434a-950b-b88803942058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:42.580691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04d8fb-8adf-434a-950b-b88803942058 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:42.581041 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04d8fb-8adf-434a-950b-b88803942058 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:42.581283 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04d8fb-8adf-434a-950b-b88803942058 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:42.593506 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd04d8fb-8adf-434a-950b-b88803942058 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:42.595114 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd04d8fb-8adf-434a-950b-b88803942058 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:42.611795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-fd04d8fb-8adf-434a-950b-b88803942058 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:42.611795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:42.611795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:42.611795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:42.611795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:42.611795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:42.611795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:42.611795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:42.611795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:42.611795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:42.611795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:42.611795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:42.611795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:42.611795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:42.611795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:42.612708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:42.613587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:42.613587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:42.613587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:42.613587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:42.613587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 267/520] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:42.614419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04d8fb-8adf-434a-950b-b88803942058 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:42.628221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21cd26-31ee-4705-9d38-66db462b9889 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:42.663740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55330695-0fdd-4383-89b6-5f0515ece236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:42.664380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55330695-0fdd-4383-89b6-5f0515ece236 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:42.664657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55330695-0fdd-4383-89b6-5f0515ece236 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:42.664931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55330695-0fdd-4383-89b6-5f0515ece236 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:42.665693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55330695-0fdd-4383-89b6-5f0515ece236 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:42.666071 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55330695-0fdd-4383-89b6-5f0515ece236 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:42.667530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55330695-0fdd-4383-89b6-5f0515ece236 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:42.690337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 254/521] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:10:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:42.690337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55330695-0fdd-4383-89b6-5f0515ece236 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:42.730744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04d8fb-8adf-434a-950b-b88803942058 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:42.765309 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf84e4-3cad-465b-92db-fb21dfe10e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:42.765979 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf84e4-3cad-465b-92db-fb21dfe10e83 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:42.766276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf84e4-3cad-465b-92db-fb21dfe10e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:42.766548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf84e4-3cad-465b-92db-fb21dfe10e83 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:42.767312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf84e4-3cad-465b-92db-fb21dfe10e83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:42.767619 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf84e4-3cad-465b-92db-fb21dfe10e83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:42.769014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf84e4-3cad-465b-92db-fb21dfe10e83 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:42.781215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 268/522] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:42.782564 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf84e4-3cad-465b-92db-fb21dfe10e83 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:42.789684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:42.799802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55330695-0fdd-4383-89b6-5f0515ece236 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:42.806874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:42.843922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6586269-f5bb-4803-b366-b42e71bf3a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:42.844591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586269-f5bb-4803-b366-b42e71bf3a2d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:42.844935 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586269-f5bb-4803-b366-b42e71bf3a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:42.845202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6586269-f5bb-4803-b366-b42e71bf3a2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:42.901862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf84e4-3cad-465b-92db-fb21dfe10e83 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:42.928455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6586269-f5bb-4803-b366-b42e71bf3a2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:42.930199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6586269-f5bb-4803-b366-b42e71bf3a2d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:42.940399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc381ca9-d4d9-4f80-9d0f-cc36e6c6a9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:42.941025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc381ca9-d4d9-4f80-9d0f-cc36e6c6a9ca None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:42.941336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc381ca9-d4d9-4f80-9d0f-cc36e6c6a9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:42.941686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc381ca9-d4d9-4f80-9d0f-cc36e6c6a9ca None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:42.942486 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc381ca9-d4d9-4f80-9d0f-cc36e6c6a9ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:42.942819 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc381ca9-d4d9-4f80-9d0f-cc36e6c6a9ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:42.944322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc381ca9-d4d9-4f80-9d0f-cc36e6c6a9ca None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:42.960932 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 269/523] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:42.961857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc381ca9-d4d9-4f80-9d0f-cc36e6c6a9ca None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:43.029172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:43.039968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 255/524] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:43.040846 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6586269-f5bb-4803-b366-b42e71bf3a2d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:43.057030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:43.082019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc381ca9-d4d9-4f80-9d0f-cc36e6c6a9ca None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:43.122688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f73b1-be72-4f50-b111-d1eb0aada9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:43.123996 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f73b1-be72-4f50-b111-d1eb0aada9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:43.124652 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f73b1-be72-4f50-b111-d1eb0aada9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:43.125176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f73b1-be72-4f50-b111-d1eb0aada9d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:43.126879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f73b1-be72-4f50-b111-d1eb0aada9d8 None admin] RBAC: Adding query filter params (role.id=624a5f7ed1cf440cb177291e89bb11ff, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=2de0ff99a2b74979a28a8b10e9ebdfcd) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:43.127201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f73b1-be72-4f50-b111-d1eb0aada9d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:43.128683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f73b1-be72-4f50-b111-d1eb0aada9d8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:43.134618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 270/525] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:10:42 2022] GET /identity/v3/role_assignments?user.id=2de0ff99a2b74979a28a8b10e9ebdfcd&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=624a5f7ed1cf440cb177291e89bb11ff => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:43.135319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f73b1-be72-4f50-b111-d1eb0aada9d8 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:43.174107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6586269-f5bb-4803-b366-b42e71bf3a2d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:43.217094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184636cf-0acf-4ac8-800a-91c36b436a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:43.218118 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184636cf-0acf-4ac8-800a-91c36b436a85 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:43.218545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184636cf-0acf-4ac8-800a-91c36b436a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:43.218996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184636cf-0acf-4ac8-800a-91c36b436a85 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:43.228114 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184636cf-0acf-4ac8-800a-91c36b436a85 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:43.230045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184636cf-0acf-4ac8-800a-91c36b436a85 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:43.244897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-184636cf-0acf-4ac8-800a-91c36b436a85 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:10:43.244897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:43.244897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:43.244897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:43.244897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:43.244897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:43.244897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:43.244897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:43.244897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:43.244897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:43.244897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:43.244897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:43.244897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:10:43.244897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:10:43.244897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:10:43.244897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:43.246090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:43.247158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:43.247158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:43.247158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:10:43.247158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:10:43.247158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:10:43.247158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:10:43.247158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:10:43.247158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:10:43.247158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:10:43.247158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:43.249680 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 256/526] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:43.250625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f73b1-be72-4f50-b111-d1eb0aada9d8 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:43.251002 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184636cf-0acf-4ac8-800a-91c36b436a85 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:43.289633 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8bf38-74ef-46f6-acf6-5c320230022a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:43.290474 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8bf38-74ef-46f6-acf6-5c320230022a None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:43.290819 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8bf38-74ef-46f6-acf6-5c320230022a None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:43.291165 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8bf38-74ef-46f6-acf6-5c320230022a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:43.298736 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8bf38-74ef-46f6-acf6-5c320230022a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:43.300219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8bf38-74ef-46f6-acf6-5c320230022a None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:43.314420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-97c8bf38-74ef-46f6-acf6-5c320230022a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:43.314420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:43.314420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:43.314420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:43.314420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:43.314420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:43.314420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:43.314420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:43.314420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:43.314420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:43.314420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:43.314420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:43.314420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:10:43.314420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:10:43.314420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:43.315307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:43.316175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:10:43.316175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:10:43.316175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:10:43.316175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:10:43.316175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:10:43.316175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:10:43.316175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:43.316175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:43.320479 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 271/527] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:43.321384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8bf38-74ef-46f6-acf6-5c320230022a None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:43.322532 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:43.326365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:43.379751 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184636cf-0acf-4ac8-800a-91c36b436a85 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:43.429545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cd28a-55b4-4d24-b758-dd5e4899fc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:43.430473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cd28a-55b4-4d24-b758-dd5e4899fc2d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:43.430920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cd28a-55b4-4d24-b758-dd5e4899fc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:43.431775 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cd28a-55b4-4d24-b758-dd5e4899fc2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:43.443623 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8bf38-74ef-46f6-acf6-5c320230022a None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:43.446099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cd28a-55b4-4d24-b758-dd5e4899fc2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:43.448618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cd28a-55b4-4d24-b758-dd5e4899fc2d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:43.469361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-598cd28a-55b4-4d24-b758-dd5e4899fc2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:43.469361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:43.469361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:43.469361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:43.469361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:43.469361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:43.469361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:43.469361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:43.469361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:43.469361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:43.469361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:43.469361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:43.469361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:43.469361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:43.469361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:43.470383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:43.471289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:43.471289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:43.471289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:43.471289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:43.477917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 257/528] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:43.477917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cd28a-55b4-4d24-b758-dd5e4899fc2d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:43.495192 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b067ad-1461-4281-b4e0-6680fbe8473e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:43.496197 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b067ad-1461-4281-b4e0-6680fbe8473e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:43.496714 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b067ad-1461-4281-b4e0-6680fbe8473e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:43.497195 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b067ad-1461-4281-b4e0-6680fbe8473e None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:43.500926 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b067ad-1461-4281-b4e0-6680fbe8473e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:43.501800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b067ad-1461-4281-b4e0-6680fbe8473e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:43.503975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b067ad-1461-4281-b4e0-6680fbe8473e None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:43.529525 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 272/529] 10.222.0.48 () {56 vars in 1139 bytes} [Mon Aug 1 15:10:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:43.530473 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b067ad-1461-4281-b4e0-6680fbe8473e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:43.593909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:43.613041 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cd28a-55b4-4d24-b758-dd5e4899fc2d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:43.638751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b067ad-1461-4281-b4e0-6680fbe8473e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:43.642620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:43.658860 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18065951-18e0-40bc-af69-5f1ce6c14f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:43.659500 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18065951-18e0-40bc-af69-5f1ce6c14f4d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:43.659784 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18065951-18e0-40bc-af69-5f1ce6c14f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:43.660258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18065951-18e0-40bc-af69-5f1ce6c14f4d None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:43.661158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18065951-18e0-40bc-af69-5f1ce6c14f4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:43.661347 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18065951-18e0-40bc-af69-5f1ce6c14f4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:43.662807 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18065951-18e0-40bc-af69-5f1ce6c14f4d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:43.680674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 258/530] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:10:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:43.681560 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18065951-18e0-40bc-af69-5f1ce6c14f4d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:43.686704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112de87-941e-4225-a9c4-cf0e40180ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:43.687667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112de87-941e-4225-a9c4-cf0e40180ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:43.688189 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112de87-941e-4225-a9c4-cf0e40180ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:43.688720 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112de87-941e-4225-a9c4-cf0e40180ed6 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:43.689645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112de87-941e-4225-a9c4-cf0e40180ed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:43.690230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112de87-941e-4225-a9c4-cf0e40180ed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:43.694205 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112de87-941e-4225-a9c4-cf0e40180ed6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:43.711297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 273/531] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:43.713015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112de87-941e-4225-a9c4-cf0e40180ed6 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:43.792045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18065951-18e0-40bc-af69-5f1ce6c14f4d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:43.828417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae72139-676f-4a39-bf50-d4d6ac2d9137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:43.828626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae72139-676f-4a39-bf50-d4d6ac2d9137 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:43.828873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae72139-676f-4a39-bf50-d4d6ac2d9137 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:43.829278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae72139-676f-4a39-bf50-d4d6ac2d9137 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:43.835526 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112de87-941e-4225-a9c4-cf0e40180ed6 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:43.859096 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:43.863087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:43.880117 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adb49d-6674-4f80-839b-963c7b22769e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:43.881056 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adb49d-6674-4f80-839b-963c7b22769e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:43.881568 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adb49d-6674-4f80-839b-963c7b22769e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:43.882091 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adb49d-6674-4f80-839b-963c7b22769e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:43.883168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adb49d-6674-4f80-839b-963c7b22769e None admin] RBAC: Adding query filter params (role.id=624a5f7ed1cf440cb177291e89bb11ff, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=94f696dfe4b34c4aa19104b5f7ee6e70) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:43.883651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adb49d-6674-4f80-839b-963c7b22769e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:43.885467 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14adb49d-6674-4f80-839b-963c7b22769e None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:43.902338 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 274/532] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:10:43 2022] GET /identity/v3/role_assignments?user.id=94f696dfe4b34c4aa19104b5f7ee6e70&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=624a5f7ed1cf440cb177291e89bb11ff => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:43.903762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adb49d-6674-4f80-839b-963c7b22769e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:43.907362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae72139-676f-4a39-bf50-d4d6ac2d9137 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:43.909004 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae72139-676f-4a39-bf50-d4d6ac2d9137 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:44.022820 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 259/533] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:44.028238 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae72139-676f-4a39-bf50-d4d6ac2d9137 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:44.129262 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adb49d-6674-4f80-839b-963c7b22769e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:44.131329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:44.171314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2e8c8-f8eb-4eed-a785-5a610fe28d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:44.172351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2e8c8-f8eb-4eed-a785-5a610fe28d5c None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:44.172854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2e8c8-f8eb-4eed-a785-5a610fe28d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:44.173519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2e8c8-f8eb-4eed-a785-5a610fe28d5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:44.174052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae72139-676f-4a39-bf50-d4d6ac2d9137 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:44.181112 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2e8c8-f8eb-4eed-a785-5a610fe28d5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:44.182879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2e8c8-f8eb-4eed-a785-5a610fe28d5c None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:44.195011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-e4d2e8c8-f8eb-4eed-a785-5a610fe28d5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:44.195011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:44.195011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:44.195011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:44.195011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:44.195011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:44.195011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:44.195011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:44.195011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:44.195011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:44.195011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:44.195011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:44.195011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:10:44.195011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:10:44.195011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:44.195886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:44.196731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:10:44.196731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:10:44.196731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:10:44.196731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:10:44.196731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:10:44.196731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:10:44.196731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:44.196731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:44.199215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 275/534] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:44.213820 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2e8c8-f8eb-4eed-a785-5a610fe28d5c None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:44.215981 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23ebb1-dd50-4559-a2a9-8f35d133815e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:44.216745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23ebb1-dd50-4559-a2a9-8f35d133815e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:44.217143 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23ebb1-dd50-4559-a2a9-8f35d133815e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:44.217548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23ebb1-dd50-4559-a2a9-8f35d133815e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:44.228602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23ebb1-dd50-4559-a2a9-8f35d133815e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:44.230272 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23ebb1-dd50-4559-a2a9-8f35d133815e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:44.247641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-ae23ebb1-dd50-4559-a2a9-8f35d133815e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:44.247641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:44.247641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:44.247641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:44.247641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:44.247641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:44.247641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:44.247641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:44.247641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:44.247641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:44.247641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:44.247641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:44.247641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:44.247641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:44.247641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:44.248744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:44.249823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:44.249823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:44.249823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:44.249823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:44.253828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 260/535] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:44.257105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23ebb1-dd50-4559-a2a9-8f35d133815e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:44.312702 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:44.383334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2e8c8-f8eb-4eed-a785-5a610fe28d5c None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:44.420782 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e48da-8b67-430e-a4c7-2a90f0791bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:44.421422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e48da-8b67-430e-a4c7-2a90f0791bee None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:44.421698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e48da-8b67-430e-a4c7-2a90f0791bee None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:44.422012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e48da-8b67-430e-a4c7-2a90f0791bee None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:44.422785 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e48da-8b67-430e-a4c7-2a90f0791bee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:44.423288 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e48da-8b67-430e-a4c7-2a90f0791bee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:44.424986 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e48da-8b67-430e-a4c7-2a90f0791bee None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:44.445302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 276/536] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:10:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:44.446640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e5bef-df3f-4273-9525-75c2d7421c16 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:44.446926 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e5bef-df3f-4273-9525-75c2d7421c16 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:44.447200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e5bef-df3f-4273-9525-75c2d7421c16 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:44.448576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 277/537] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:44.451739 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e5bef-df3f-4273-9525-75c2d7421c16 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:44.459119 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23ebb1-dd50-4559-a2a9-8f35d133815e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:44.481070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:44.503656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7f24f-7950-4a79-9065-0150da4b67cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:44.504404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7f24f-7950-4a79-9065-0150da4b67cc None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:44.504679 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7f24f-7950-4a79-9065-0150da4b67cc None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:44.504948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7f24f-7950-4a79-9065-0150da4b67cc None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:44.505655 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7f24f-7950-4a79-9065-0150da4b67cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:44.505984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7f24f-7950-4a79-9065-0150da4b67cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:44.507453 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7f24f-7950-4a79-9065-0150da4b67cc None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:44.520891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 261/538] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:44.522265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7efad-701a-42a7-89e2-da9434b4dc49 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:44.522546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7efad-701a-42a7-89e2-da9434b4dc49 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:44.522816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7efad-701a-42a7-89e2-da9434b4dc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:44.542313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-8ac7efad-701a-42a7-89e2-da9434b4dc49 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:44.543525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8ac7efad-701a-42a7-89e2-da9434b4dc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:44.580386 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e5bef-df3f-4273-9525-75c2d7421c16 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:44.589486 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:44.625193 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14203753-0757-42af-82c4-15a66333749f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:44.625884 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14203753-0757-42af-82c4-15a66333749f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:44.626221 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14203753-0757-42af-82c4-15a66333749f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:44.626494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14203753-0757-42af-82c4-15a66333749f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:44.627512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14203753-0757-42af-82c4-15a66333749f None admin] RBAC: Adding query filter params (role.id=624a5f7ed1cf440cb177291e89bb11ff, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=63824ba9369c4899a4b946588cbdae24) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:44.627870 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14203753-0757-42af-82c4-15a66333749f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:44.629339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14203753-0757-42af-82c4-15a66333749f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:44.633920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 262/539] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:44.635919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7efad-701a-42a7-89e2-da9434b4dc49 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:44.639605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 278/540] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:10:44 2022] GET /identity/v3/role_assignments?user.id=63824ba9369c4899a4b946588cbdae24&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=624a5f7ed1cf440cb177291e89bb11ff => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:44.656230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14203753-0757-42af-82c4-15a66333749f None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:44.754351 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7efad-701a-42a7-89e2-da9434b4dc49 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:44.762788 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:44.794235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1c492-fec7-46ed-bd35-c8182c4788a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:44.794822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1c492-fec7-46ed-bd35-c8182c4788a9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:44.795098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1c492-fec7-46ed-bd35-c8182c4788a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:44.795371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d1c492-fec7-46ed-bd35-c8182c4788a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:44.801153 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14203753-0757-42af-82c4-15a66333749f None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:44.803363 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d1c492-fec7-46ed-bd35-c8182c4788a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:44.805285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d1c492-fec7-46ed-bd35-c8182c4788a9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:44.818401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-52d1c492-fec7-46ed-bd35-c8182c4788a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:44.818401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:44.818401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:44.818401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:44.818401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:44.818401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:44.818401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:44.818401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:44.818401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:44.818401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:44.818401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:44.818401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:44.818401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:10:44.818401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:10:44.818401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:44.819502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:44.820527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:10:44.820527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:10:44.820527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:10:44.820527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:10:44.820527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:10:44.820527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:10:44.820527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:44.820527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:44.822850 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 263/541] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:44.828609 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1c492-fec7-46ed-bd35-c8182c4788a9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:44.866356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b8d3f-2a87-48a4-b9b6-cc166bbea281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:44.867017 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b8d3f-2a87-48a4-b9b6-cc166bbea281 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:44.867299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b8d3f-2a87-48a4-b9b6-cc166bbea281 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:44.867571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b8d3f-2a87-48a4-b9b6-cc166bbea281 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:44.879310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b8d3f-2a87-48a4-b9b6-cc166bbea281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:44.884413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-530b8d3f-2a87-48a4-b9b6-cc166bbea281 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:44.884413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:44.884413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:44.884413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:44.884413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:44.884413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:44.884413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:44.884413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:44.884413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:44.884413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:44.884413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:44.884413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:44.884413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:44.884413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:44.884413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:44.884413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:44.885482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:44.885482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:44.885482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:44.885482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:44.885482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:44.885482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:44.885482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:44.885482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:44.885482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:44.885482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:44.885482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:44.885482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:44.885482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:44.885482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:44.885482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:44.885482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:44.886645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:44.886645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:44.886645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:44.886645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:44.886645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:44.886645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:44.886645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:44.886645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:44.886645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:44.886645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:44.886645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:44.886645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:44.886645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 279/542] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:44.891825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b8d3f-2a87-48a4-b9b6-cc166bbea281 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:44.933437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:44.985949 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d1c492-fec7-46ed-bd35-c8182c4788a9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:45.029804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5986c-a1d4-41a2-a419-646c5b281173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:45.029804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5986c-a1d4-41a2-a419-646c5b281173 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:45.029804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5986c-a1d4-41a2-a419-646c5b281173 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:45.029804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5986c-a1d4-41a2-a419-646c5b281173 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:45.030223 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5986c-a1d4-41a2-a419-646c5b281173 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:45.030223 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5986c-a1d4-41a2-a419-646c5b281173 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:45.030576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5986c-a1d4-41a2-a419-646c5b281173 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:45.051054 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 264/543] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:10:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:45.057383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5986c-a1d4-41a2-a419-646c5b281173 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:45.089558 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b8d3f-2a87-48a4-b9b6-cc166bbea281 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:45.113519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:45.158115 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c7969-a9df-4540-9040-320da25576ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:45.158751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c7969-a9df-4540-9040-320da25576ef None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:45.158944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c7969-a9df-4540-9040-320da25576ef None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:45.159232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c7969-a9df-4540-9040-320da25576ef None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:45.159985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c7969-a9df-4540-9040-320da25576ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:45.160295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c7969-a9df-4540-9040-320da25576ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:45.164107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c7969-a9df-4540-9040-320da25576ef None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:45.175883 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 280/544] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:10:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:45.182412 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c7969-a9df-4540-9040-320da25576ef None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:45.255023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5986c-a1d4-41a2-a419-646c5b281173 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:45.291958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:45.298841 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c7969-a9df-4540-9040-320da25576ef None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:45.326546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf83510-9e4b-4aab-ab94-24f37faaa36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:45.327198 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf83510-9e4b-4aab-ab94-24f37faaa36c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:45.327475 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf83510-9e4b-4aab-ab94-24f37faaa36c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:45.327746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf83510-9e4b-4aab-ab94-24f37faaa36c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:45.328565 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf83510-9e4b-4aab-ab94-24f37faaa36c None admin] RBAC: Adding query filter params (role.id=624a5f7ed1cf440cb177291e89bb11ff, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=5ef6577b10f7426a925d594e627f27cf) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:45.328870 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf83510-9e4b-4aab-ab94-24f37faaa36c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:45.330424 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf83510-9e4b-4aab-ab94-24f37faaa36c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:45.342562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 265/545] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:10:45 2022] GET /identity/v3/role_assignments?user.id=5ef6577b10f7426a925d594e627f27cf&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=624a5f7ed1cf440cb177291e89bb11ff => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:45.356704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6109392-d05d-490c-9969-50911fae283c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:45.357399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6109392-d05d-490c-9969-50911fae283c None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:45.358085 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6109392-d05d-490c-9969-50911fae283c None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:45.358363 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6109392-d05d-490c-9969-50911fae283c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:45.375179 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6109392-d05d-490c-9969-50911fae283c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:45.376843 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6109392-d05d-490c-9969-50911fae283c None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:45.393955 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0b14b-e965-408c-936b-8c501c609657 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:45.394300 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0b14b-e965-408c-936b-8c501c609657 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:45.394583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0b14b-e965-408c-936b-8c501c609657 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:45.395083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-d6109392-d05d-490c-9969-50911fae283c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:45.395083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:45.395083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:45.395083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:45.395083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:45.395083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:45.395083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:45.395083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:45.395083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:45.395083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:45.395083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:45.395083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:45.395083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:45.395083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:45.395083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:45.395941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:45.396786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:45.396786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:45.396786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:45.396786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:45.396993 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 266/546] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:10:45.399105 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 281/547] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:45.402976 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c1f4a-39ea-443d-8cf7-bc01872536cb None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:45.403277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c1f4a-39ea-443d-8cf7-bc01872536cb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:45.403550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c1f4a-39ea-443d-8cf7-bc01872536cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:45.407430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6109392-d05d-490c-9969-50911fae283c None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:45.439255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f52c1f4a-39ea-443d-8cf7-bc01872536cb None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:45.440415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f52c1f4a-39ea-443d-8cf7-bc01872536cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:45.571675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6109392-d05d-490c-9969-50911fae283c None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:45.584319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 267/548] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:45.616873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c1f4a-39ea-443d-8cf7-bc01872536cb None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:45.623537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9cc036-251f-4ca8-b166-3bfb4dad7601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:45.624463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cc036-251f-4ca8-b166-3bfb4dad7601 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:45.624934 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cc036-251f-4ca8-b166-3bfb4dad7601 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:45.625504 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cc036-251f-4ca8-b166-3bfb4dad7601 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:45.626480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9cc036-251f-4ca8-b166-3bfb4dad7601 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:45.627107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9cc036-251f-4ca8-b166-3bfb4dad7601 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:45.628692 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9cc036-251f-4ca8-b166-3bfb4dad7601 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:45.640328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:45.654388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 282/549] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:45.660030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9cc036-251f-4ca8-b166-3bfb4dad7601 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:45.662927 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:45.732383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c1f4a-39ea-443d-8cf7-bc01872536cb None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:45.770720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19725bbc-73a2-4f91-8b3f-0ce0bd1ae27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:45.772073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19725bbc-73a2-4f91-8b3f-0ce0bd1ae27c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:45.772552 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19725bbc-73a2-4f91-8b3f-0ce0bd1ae27c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:45.773061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19725bbc-73a2-4f91-8b3f-0ce0bd1ae27c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:45.785389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19725bbc-73a2-4f91-8b3f-0ce0bd1ae27c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:45.790926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-19725bbc-73a2-4f91-8b3f-0ce0bd1ae27c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:45.790926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:45.790926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:45.790926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:45.790926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:45.790926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:45.790926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:45.790926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:45.790926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:45.790926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:45.790926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:45.790926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:45.790926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:45.790926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:45.790926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:45.790926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:45.792321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:45.792321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:45.792321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:45.792321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:45.792321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:45.792321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:45.792321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:45.792321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:45.792321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:45.792321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:45.792321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:45.792321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:45.792321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:45.792321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:45.792321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:45.792321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:45.793334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:45.793334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:45.793334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:45.793334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:45.793334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:45.793334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:45.793334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:45.793334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:45.793334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:45.793334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:45.793334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:45.793334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:45.796242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 268/550] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:45.802669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19725bbc-73a2-4f91-8b3f-0ce0bd1ae27c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:45.922326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:45.926323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19725bbc-73a2-4f91-8b3f-0ce0bd1ae27c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:45.965087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e3dad-f69f-43c3-87e2-a3b7c21d2373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:45.966162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e3dad-f69f-43c3-87e2-a3b7c21d2373 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:45.966660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e3dad-f69f-43c3-87e2-a3b7c21d2373 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:45.967128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e3dad-f69f-43c3-87e2-a3b7c21d2373 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:45.968093 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e3dad-f69f-43c3-87e2-a3b7c21d2373 None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:45.968591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e3dad-f69f-43c3-87e2-a3b7c21d2373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:45.970335 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994e3dad-f69f-43c3-87e2-a3b7c21d2373 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:45.977998 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9cc036-251f-4ca8-b166-3bfb4dad7601 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:45.987155 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 269/551] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:10:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:45.989670 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e3dad-f69f-43c3-87e2-a3b7c21d2373 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:46.044518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461d122-05a4-483f-b47f-f79bdc21aa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:46.045252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461d122-05a4-483f-b47f-f79bdc21aa4d None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:46.045528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461d122-05a4-483f-b47f-f79bdc21aa4d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:46.045810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461d122-05a4-483f-b47f-f79bdc21aa4d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:46.065057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461d122-05a4-483f-b47f-f79bdc21aa4d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:46.066596 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461d122-05a4-483f-b47f-f79bdc21aa4d None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:46.099274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-6461d122-05a4-483f-b47f-f79bdc21aa4d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:10:46.099274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:46.099274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:46.099274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:46.099274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:46.099274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:46.099274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:46.099274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:46.099274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:46.099274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:46.099274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:46.099274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:46.099274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:10:46.099274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:10:46.099274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:10:46.099274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:46.100364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:46.101370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:46.101370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:46.101370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:10:46.101370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:10:46.101370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:10:46.101370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:10:46.101370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:10:46.101370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:10:46.101370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:10:46.101370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:46.101370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 283/552] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:10:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 441 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:46.106845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461d122-05a4-483f-b47f-f79bdc21aa4d None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:46.111673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e3dad-f69f-43c3-87e2-a3b7c21d2373 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:46.149389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a39678-73de-4ee0-8fd3-7267a358e3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:46.150467 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a39678-73de-4ee0-8fd3-7267a358e3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:46.151043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a39678-73de-4ee0-8fd3-7267a358e3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:46.151762 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a39678-73de-4ee0-8fd3-7267a358e3a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:46.164306 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a39678-73de-4ee0-8fd3-7267a358e3a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:46.166230 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a39678-73de-4ee0-8fd3-7267a358e3a4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:46.184022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-26a39678-73de-4ee0-8fd3-7267a358e3a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:46.184022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:46.184022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:46.184022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:46.184022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:46.184022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:46.184022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:46.184022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:46.184022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:46.184022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:46.184022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:46.184022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:46.184022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:46.184022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:46.184022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:46.184962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:46.185855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:46.185855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:46.185855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:46.185855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:46.188279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:46.188875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 270/553] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:46.194517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a39678-73de-4ee0-8fd3-7267a358e3a4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:46.259447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:46.305688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461d122-05a4-483f-b47f-f79bdc21aa4d None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:46.346092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a39678-73de-4ee0-8fd3-7267a358e3a4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:46.352861 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf6720-54a9-4522-b376-afdd5352e2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:46.354219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf6720-54a9-4522-b376-afdd5352e2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:46.354670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf6720-54a9-4522-b376-afdd5352e2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:46.355107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf6720-54a9-4522-b376-afdd5352e2f2 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:46.356513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf6720-54a9-4522-b376-afdd5352e2f2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:46.357392 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf6720-54a9-4522-b376-afdd5352e2f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:46.359664 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf6720-54a9-4522-b376-afdd5352e2f2 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:46.382431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87c9f4-764a-48b5-bccf-24516c40f05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:46.383250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87c9f4-764a-48b5-bccf-24516c40f05f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:46.387898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87c9f4-764a-48b5-bccf-24516c40f05f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:46.387898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87c9f4-764a-48b5-bccf-24516c40f05f None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:46.387898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87c9f4-764a-48b5-bccf-24516c40f05f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:46.387898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87c9f4-764a-48b5-bccf-24516c40f05f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:46.387898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87c9f4-764a-48b5-bccf-24516c40f05f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:46.388559 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 284/554] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 15:10:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:46.390606 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0651673-d26f-4d60-906d-19d295a306e0 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:46.390891 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0651673-d26f-4d60-906d-19d295a306e0 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:46.391165 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0651673-d26f-4d60-906d-19d295a306e0 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:46.394034 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 285/555] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:46.394935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0651673-d26f-4d60-906d-19d295a306e0 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:46.412940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 271/556] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:46.412940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7018244-955a-45ff-b68b-cba2727a6085 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:46.412940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7018244-955a-45ff-b68b-cba2727a6085 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:46.412940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7018244-955a-45ff-b68b-cba2727a6085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:46.433378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a7018244-955a-45ff-b68b-cba2727a6085 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:46.434627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a7018244-955a-45ff-b68b-cba2727a6085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:46.498415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:46.519587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 272/557] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:46.520747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7018244-955a-45ff-b68b-cba2727a6085 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:46.535583 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0651673-d26f-4d60-906d-19d295a306e0 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:46.568551 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:46.586457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762c81a-2376-475b-b41d-ec7e5ca9a806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:46.587109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762c81a-2376-475b-b41d-ec7e5ca9a806 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:46.587537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762c81a-2376-475b-b41d-ec7e5ca9a806 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:46.587704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762c81a-2376-475b-b41d-ec7e5ca9a806 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:46.631996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7018244-955a-45ff-b68b-cba2727a6085 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:46.660176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762c81a-2376-475b-b41d-ec7e5ca9a806 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:46.661709 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762c81a-2376-475b-b41d-ec7e5ca9a806 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:46.669909 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8207019-50dc-4993-bcb7-11c381cedf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:46.670798 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8207019-50dc-4993-bcb7-11c381cedf62 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:46.671212 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8207019-50dc-4993-bcb7-11c381cedf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:46.671649 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8207019-50dc-4993-bcb7-11c381cedf62 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:46.680380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8207019-50dc-4993-bcb7-11c381cedf62 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:46.682273 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8207019-50dc-4993-bcb7-11c381cedf62 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:46.696633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-a8207019-50dc-4993-bcb7-11c381cedf62 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:10:46.696633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:46.696633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:46.696633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:46.696633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:46.696633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:46.696633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:46.696633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:46.696633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:46.696633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:46.696633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:46.696633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:46.696633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:10:46.696633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:10:46.696633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:10:46.696633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:46.698298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:46.699490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:46.699490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:46.699490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:10:46.699490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:10:46.699490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:10:46.699490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:10:46.699490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:10:46.699490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:10:46.699490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:10:46.699490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:46.701751 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 273/558] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 15:10:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:46.707064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8207019-50dc-4993-bcb7-11c381cedf62 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:46.757171 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:46.774030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 286/559] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:46.774932 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762c81a-2376-475b-b41d-ec7e5ca9a806 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:46.798450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:46.825440 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8207019-50dc-4993-bcb7-11c381cedf62 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:46.861967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f52e2-4320-4c3e-979c-601c18f8dc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:46.862863 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f52e2-4320-4c3e-979c-601c18f8dc52 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:46.863278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f52e2-4320-4c3e-979c-601c18f8dc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:46.863769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f52e2-4320-4c3e-979c-601c18f8dc52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:46.875204 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f52e2-4320-4c3e-979c-601c18f8dc52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:46.879747 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762c81a-2376-475b-b41d-ec7e5ca9a806 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:46.881549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-1e8f52e2-4320-4c3e-979c-601c18f8dc52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:46.881549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:46.881549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:46.881549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:46.881549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:46.881549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:46.881549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:46.881549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:46.881549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:46.881549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:46.881549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:46.881549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:46.881549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:46.881549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:46.881549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:46.881549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:46.882770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:46.882770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:46.882770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:46.882770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:46.882770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:46.882770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:46.882770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:46.882770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:46.882770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:46.882770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:46.882770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:46.882770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:46.882770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:46.882770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:46.882770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:46.882770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:46.884172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:46.884172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:46.884172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:46.884172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:46.884172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:46.884172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:46.884172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:46.884172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:46.884172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:46.884172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:46.884172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:46.884172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:46.886410 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 274/560] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:46.887450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f52e2-4320-4c3e-979c-601c18f8dc52 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:46.922473 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e641c69-1332-4fe0-9c99-522068b6d5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:46.923199 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e641c69-1332-4fe0-9c99-522068b6d5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:46.923485 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e641c69-1332-4fe0-9c99-522068b6d5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:46.923811 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e641c69-1332-4fe0-9c99-522068b6d5a6 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:46.924629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e641c69-1332-4fe0-9c99-522068b6d5a6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:46.924985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e641c69-1332-4fe0-9c99-522068b6d5a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:46.926634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e641c69-1332-4fe0-9c99-522068b6d5a6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:46.951041 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 287/561] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 15:10:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:46.951041 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e641c69-1332-4fe0-9c99-522068b6d5a6 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:47.010941 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f52e2-4320-4c3e-979c-601c18f8dc52 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:47.013844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:47.050616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:47.055563 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fa460-f83c-4601-a12f-4f573d384a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:47.056442 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fa460-f83c-4601-a12f-4f573d384a78 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:47.056881 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fa460-f83c-4601-a12f-4f573d384a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:47.057151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fa460-f83c-4601-a12f-4f573d384a78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:47.068368 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fa460-f83c-4601-a12f-4f573d384a78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:47.069773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fa460-f83c-4601-a12f-4f573d384a78 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:47.073008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e641c69-1332-4fe0-9c99-522068b6d5a6 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:47.085948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d34fa460-f83c-4601-a12f-4f573d384a78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:47.085948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:47.085948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:47.085948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:47.085948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:47.085948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:47.085948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:47.085948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:47.085948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:47.085948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:47.085948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:47.085948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:47.085948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:47.085948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:47.085948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:47.086932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:47.087814 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:47.087814 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:47.087814 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:47.087814 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:47.087814 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 275/562] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:47.088103 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fa460-f83c-4601-a12f-4f573d384a78 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:47.114613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a907e8-26ea-4986-9842-865796aca062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:47.115257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a907e8-26ea-4986-9842-865796aca062 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:47.115538 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a907e8-26ea-4986-9842-865796aca062 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:47.115807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a907e8-26ea-4986-9842-865796aca062 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:47.116535 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a907e8-26ea-4986-9842-865796aca062 None admin] RBAC: Adding query filter params (name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:47.116843 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a907e8-26ea-4986-9842-865796aca062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:47.118264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a907e8-26ea-4986-9842-865796aca062 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:47.127347 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 288/563] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:10:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:47.128484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946a459-6e84-4d5d-b51a-8388c19ce1dc None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:47.128771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946a459-6e84-4d5d-b51a-8388c19ce1dc None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:47.129045 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946a459-6e84-4d5d-b51a-8388c19ce1dc None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:47.133924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 289/564] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:47.135141 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946a459-6e84-4d5d-b51a-8388c19ce1dc None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:47.216292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fa460-f83c-4601-a12f-4f573d384a78 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:47.239422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946a459-6e84-4d5d-b51a-8388c19ce1dc None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:47.259805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0144b0-0bc6-4ece-a4f6-b137ac9e9633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:47.261206 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0144b0-0bc6-4ece-a4f6-b137ac9e9633 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:47.261874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0144b0-0bc6-4ece-a4f6-b137ac9e9633 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:47.262352 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0144b0-0bc6-4ece-a4f6-b137ac9e9633 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:47.274469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9d860-4891-4e42-94c6-f09ca10b5420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:47.275187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9d860-4891-4e42-94c6-f09ca10b5420 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:47.275381 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9d860-4891-4e42-94c6-f09ca10b5420 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:47.275700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9d860-4891-4e42-94c6-f09ca10b5420 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:47.276364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9d860-4891-4e42-94c6-f09ca10b5420 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:47.276672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9d860-4891-4e42-94c6-f09ca10b5420 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:47.276988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0144b0-0bc6-4ece-a4f6-b137ac9e9633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:47.278085 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9d860-4891-4e42-94c6-f09ca10b5420 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:47.278781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0144b0-0bc6-4ece-a4f6-b137ac9e9633 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:47.287346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:47.290942 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 290/565] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:47.296268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-7c0144b0-0bc6-4ece-a4f6-b137ac9e9633 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:47.296268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:47.296268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:47.296268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:47.296268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:47.296268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:47.296268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:47.296268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:47.296268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:47.296268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:47.296268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:47.296268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:47.296268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:47.296268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:47.296268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:47.297294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:47.298420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:47.298420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:47.298420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:47.298420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:47.298637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9d860-4891-4e42-94c6-f09ca10b5420 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:47.303923 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 276/566] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:47.303923 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72488bdd-99c2-4b4c-9faa-5b998c3df0bc None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:47.303923 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72488bdd-99c2-4b4c-9faa-5b998c3df0bc None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:47.303923 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72488bdd-99c2-4b4c-9faa-5b998c3df0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:47.320274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-72488bdd-99c2-4b4c-9faa-5b998c3df0bc None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:47.321596 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-72488bdd-99c2-4b4c-9faa-5b998c3df0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:47.324622 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:47.397660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 277/567] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:47.399258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72488bdd-99c2-4b4c-9faa-5b998c3df0bc None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:47.411589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9d860-4891-4e42-94c6-f09ca10b5420 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:47.448436 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570f014-829b-4df7-a087-98b7633a9d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:47.449047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570f014-829b-4df7-a087-98b7633a9d3e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:47.449394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570f014-829b-4df7-a087-98b7633a9d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:47.449665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570f014-829b-4df7-a087-98b7633a9d3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:47.460529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570f014-829b-4df7-a087-98b7633a9d3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:47.462017 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570f014-829b-4df7-a087-98b7633a9d3e None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:47.478333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-a570f014-829b-4df7-a087-98b7633a9d3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:47.478333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:47.478333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:47.478333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:47.478333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:47.478333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:47.478333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:47.478333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:47.478333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:47.478333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:47.478333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:47.478333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:47.478333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:47.478333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:47.478333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:47.479419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:47.480319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:47.480319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:47.480319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:47.480319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:47.480319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 291/568] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:47.480319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570f014-829b-4df7-a087-98b7633a9d3e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:47.519468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72488bdd-99c2-4b4c-9faa-5b998c3df0bc None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:47.537335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:47.555824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32b87f-567d-49a7-87d9-90b3ae32c350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:47.556399 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32b87f-567d-49a7-87d9-90b3ae32c350 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:47.556674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32b87f-567d-49a7-87d9-90b3ae32c350 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:47.556944 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32b87f-567d-49a7-87d9-90b3ae32c350 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:47.565116 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32b87f-567d-49a7-87d9-90b3ae32c350 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:47.566603 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32b87f-567d-49a7-87d9-90b3ae32c350 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:47.568448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:47.579615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-2e32b87f-567d-49a7-87d9-90b3ae32c350 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:47.579615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:47.579615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:47.579615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:47.579615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:47.579615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:47.579615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:47.579615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:47.579615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:47.579615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:47.579615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:47.579615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:47.579615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:10:47.579615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:10:47.579615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:47.580517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:47.581522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:10:47.581522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:10:47.581522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:10:47.581522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:10:47.581522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:10:47.581522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:10:47.581522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:47.581522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:47.581522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 278/569] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:47.582140 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32b87f-567d-49a7-87d9-90b3ae32c350 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:47.594325 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570f014-829b-4df7-a087-98b7633a9d3e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:47.632790 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a396e25-193a-4049-a2be-f5e3dc67be86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:47.633420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a396e25-193a-4049-a2be-f5e3dc67be86 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:47.633700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a396e25-193a-4049-a2be-f5e3dc67be86 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:47.634003 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a396e25-193a-4049-a2be-f5e3dc67be86 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:47.634700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a396e25-193a-4049-a2be-f5e3dc67be86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:47.635013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a396e25-193a-4049-a2be-f5e3dc67be86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:47.636541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a396e25-193a-4049-a2be-f5e3dc67be86 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:47.649479 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 292/570] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:47.650141 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a396e25-193a-4049-a2be-f5e3dc67be86 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:47.686495 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32b87f-567d-49a7-87d9-90b3ae32c350 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:47.720050 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef88ca-9cfb-48f4-8b3a-8bd529991e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:47.720695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef88ca-9cfb-48f4-8b3a-8bd529991e13 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:47.720970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef88ca-9cfb-48f4-8b3a-8bd529991e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:47.721240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef88ca-9cfb-48f4-8b3a-8bd529991e13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:47.732008 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef88ca-9cfb-48f4-8b3a-8bd529991e13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:47.737476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-46ef88ca-9cfb-48f4-8b3a-8bd529991e13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:47.737476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:47.737476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:47.737476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:47.737476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:47.737476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:47.737476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:47.737476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:47.737476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:47.737476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:47.737476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:47.737476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:47.737476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:47.737476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:47.737476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:47.737476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:47.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:47.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:47.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:47.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:47.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:47.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:47.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:47.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:47.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:47.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:47.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:47.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:47.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:47.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:47.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:47.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:47.739628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:47.739628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:47.739628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:47.739628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:47.739628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:47.739628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:47.739628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:47.739628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:47.739628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:47.739628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:47.739628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:47.739628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:47.739628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 279/571] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:47.739628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef88ca-9cfb-48f4-8b3a-8bd529991e13 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:47.754028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a396e25-193a-4049-a2be-f5e3dc67be86 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:47.768104 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:47.790115 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31deada0-8376-4ca2-b96a-389949296a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:47.790728 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31deada0-8376-4ca2-b96a-389949296a29 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:47.790998 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31deada0-8376-4ca2-b96a-389949296a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:47.791262 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31deada0-8376-4ca2-b96a-389949296a29 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:47.791950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31deada0-8376-4ca2-b96a-389949296a29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:47.792332 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31deada0-8376-4ca2-b96a-389949296a29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:47.794233 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31deada0-8376-4ca2-b96a-389949296a29 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:47.799436 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:47.806362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 293/572] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:47.807793 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81145a-5ec1-4127-b4d1-9b9e6da38c3b None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:47.808079 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81145a-5ec1-4127-b4d1-9b9e6da38c3b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:47.808349 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81145a-5ec1-4127-b4d1-9b9e6da38c3b None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:47.809568 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 294/573] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:47.810288 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81145a-5ec1-4127-b4d1-9b9e6da38c3b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:47.848082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef88ca-9cfb-48f4-8b3a-8bd529991e13 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:47.889990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432a31a-8147-4683-b3c6-4c2b2a5115b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:47.890632 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432a31a-8147-4683-b3c6-4c2b2a5115b9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:47.890915 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432a31a-8147-4683-b3c6-4c2b2a5115b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:47.891187 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432a31a-8147-4683-b3c6-4c2b2a5115b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:47.892075 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432a31a-8147-4683-b3c6-4c2b2a5115b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:47.892381 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432a31a-8147-4683-b3c6-4c2b2a5115b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:47.893861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432a31a-8147-4683-b3c6-4c2b2a5115b9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:47.905614 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 280/574] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:10:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:47.906451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432a31a-8147-4683-b3c6-4c2b2a5115b9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:47.920783 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81145a-5ec1-4127-b4d1-9b9e6da38c3b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:47.954201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26996297-efa0-4d61-b103-d1c699f7968f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:47.954844 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26996297-efa0-4d61-b103-d1c699f7968f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:47.955116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26996297-efa0-4d61-b103-d1c699f7968f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:47.955391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26996297-efa0-4d61-b103-d1c699f7968f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:47.961950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26996297-efa0-4d61-b103-d1c699f7968f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:47.963391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26996297-efa0-4d61-b103-d1c699f7968f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:47.974845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-26996297-efa0-4d61-b103-d1c699f7968f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:47.974845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:47.974845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:47.974845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:47.974845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:47.974845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:47.974845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:47.974845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:47.974845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:47.974845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:47.974845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:47.974845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:47.974845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:10:47.974845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:10:47.974845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:47.975910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:47.976858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:10:47.976858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:10:47.976858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:10:47.976858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:10:47.976858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:10:47.976858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:10:47.976858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:47.976858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:47.976858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 295/575] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:47.977360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26996297-efa0-4d61-b103-d1c699f7968f None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:48.007347 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432a31a-8147-4683-b3c6-4c2b2a5115b9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:48.009257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:48.024386 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:48.041191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58675050-fa7e-4181-9cb0-37e994ec1612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:48.041974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58675050-fa7e-4181-9cb0-37e994ec1612 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:48.042474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58675050-fa7e-4181-9cb0-37e994ec1612 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:48.042820 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58675050-fa7e-4181-9cb0-37e994ec1612 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:48.043546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58675050-fa7e-4181-9cb0-37e994ec1612 None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:48.043849 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58675050-fa7e-4181-9cb0-37e994ec1612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:48.045244 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58675050-fa7e-4181-9cb0-37e994ec1612 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:48.053563 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 281/576] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:10:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:48.054759 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a3708-04ea-4fc5-8581-4b2ea1e06881 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:48.055045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a3708-04ea-4fc5-8581-4b2ea1e06881 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:48.055319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a3708-04ea-4fc5-8581-4b2ea1e06881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:48.072450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-851a3708-04ea-4fc5-8581-4b2ea1e06881 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:48.073719 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-851a3708-04ea-4fc5-8581-4b2ea1e06881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:48.079861 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26996297-efa0-4d61-b103-d1c699f7968f None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:48.114074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16b894-b212-4e69-9a51-3f089e2be0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:48.114727 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16b894-b212-4e69-9a51-3f089e2be0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:48.115009 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16b894-b212-4e69-9a51-3f089e2be0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:48.115285 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16b894-b212-4e69-9a51-3f089e2be0a3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:48.123157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16b894-b212-4e69-9a51-3f089e2be0a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:48.124613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16b894-b212-4e69-9a51-3f089e2be0a3 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:48.135878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-7f16b894-b212-4e69-9a51-3f089e2be0a3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:10:48.135878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:48.135878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:48.135878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:48.135878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:48.135878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:48.135878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:48.135878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:48.135878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:48.135878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:48.135878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:48.135878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:48.135878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:10:48.135878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:10:48.135878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:10:48.135878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:48.136987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:48.138013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:48.138013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:48.138013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:10:48.138013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:10:48.138013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:10:48.138013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:10:48.138013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:10:48.138013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:10:48.138013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:10:48.138013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:48.138013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 296/577] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:48.138013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16b894-b212-4e69-9a51-3f089e2be0a3 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:48.143986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 282/578] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:48.144834 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a3708-04ea-4fc5-8581-4b2ea1e06881 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:48.232322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:48.242254 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16b894-b212-4e69-9a51-3f089e2be0a3 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:48.259623 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a3708-04ea-4fc5-8581-4b2ea1e06881 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:48.270828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:48.278490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce6bf7-5cab-404f-a43e-fbe93d37abd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:48.279112 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce6bf7-5cab-404f-a43e-fbe93d37abd6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:48.279388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce6bf7-5cab-404f-a43e-fbe93d37abd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:48.279658 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce6bf7-5cab-404f-a43e-fbe93d37abd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:48.280455 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce6bf7-5cab-404f-a43e-fbe93d37abd6 None admin] RBAC: Adding query filter params (role.id=624a5f7ed1cf440cb177291e89bb11ff, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=25a26f54accf4264aef646c3757de7e9) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:48.280764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce6bf7-5cab-404f-a43e-fbe93d37abd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:48.282288 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce6bf7-5cab-404f-a43e-fbe93d37abd6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:48.287577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 297/579] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:10:48 2022] GET /identity/v3/role_assignments?user.id=25a26f54accf4264aef646c3757de7e9&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=624a5f7ed1cf440cb177291e89bb11ff => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:48.288344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce6bf7-5cab-404f-a43e-fbe93d37abd6 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:48.294324 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b81967-f6ff-4964-8267-f3fe513e51e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:48.294974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b81967-f6ff-4964-8267-f3fe513e51e5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:48.295252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b81967-f6ff-4964-8267-f3fe513e51e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:48.295524 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b81967-f6ff-4964-8267-f3fe513e51e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:48.296299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b81967-f6ff-4964-8267-f3fe513e51e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:48.296606 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b81967-f6ff-4964-8267-f3fe513e51e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:48.298188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b81967-f6ff-4964-8267-f3fe513e51e5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:48.311525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 283/580] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:10:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:48.317685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b81967-f6ff-4964-8267-f3fe513e51e5 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:48.408130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce6bf7-5cab-404f-a43e-fbe93d37abd6 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:48.430790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b81967-f6ff-4964-8267-f3fe513e51e5 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:48.449173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7af52f-c1e3-43b2-a7bb-7919b0f548a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:48.449866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7af52f-c1e3-43b2-a7bb-7919b0f548a2 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:48.450678 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7af52f-c1e3-43b2-a7bb-7919b0f548a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:48.450678 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7af52f-c1e3-43b2-a7bb-7919b0f548a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:48.467289 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7af52f-c1e3-43b2-a7bb-7919b0f548a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:48.468781 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7af52f-c1e3-43b2-a7bb-7919b0f548a2 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:48.469644 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e6300-cd07-4631-8cf1-d8f251d3c4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:48.470317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e6300-cd07-4631-8cf1-d8f251d3c4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:48.470583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e6300-cd07-4631-8cf1-d8f251d3c4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:48.470853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e6300-cd07-4631-8cf1-d8f251d3c4c2 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:48.471651 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e6300-cd07-4631-8cf1-d8f251d3c4c2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:48.471950 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e6300-cd07-4631-8cf1-d8f251d3c4c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:48.473389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e6300-cd07-4631-8cf1-d8f251d3c4c2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:48.485508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-ef7af52f-c1e3-43b2-a7bb-7919b0f548a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:48.485508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:48.485508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:48.485508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:48.485508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:48.485508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:48.485508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:48.485508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:48.485508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:48.485508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:48.485508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:48.485508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:48.485508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:48.485508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:48.485508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:48.486469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:48.487466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:48.487466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:48.487466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:48.487466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:48.487684 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 298/581] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:48.488423 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7af52f-c1e3-43b2-a7bb-7919b0f548a2 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:48.494465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 284/582] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 15:10:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:48.495554 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e6300-cd07-4631-8cf1-d8f251d3c4c2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:48.500951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:48.525171 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:48.601039 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7af52f-c1e3-43b2-a7bb-7919b0f548a2 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:48.604435 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e6300-cd07-4631-8cf1-d8f251d3c4c2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:48.636489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744684c-21b2-4b99-a524-c88677bf2ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:48.637131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744684c-21b2-4b99-a524-c88677bf2ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:48.637413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744684c-21b2-4b99-a524-c88677bf2ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:48.637677 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744684c-21b2-4b99-a524-c88677bf2ee9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:48.639793 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2673b14-4970-469b-bc01-5f3d581ce75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:48.640477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2673b14-4970-469b-bc01-5f3d581ce75b None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:48.640757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2673b14-4970-469b-bc01-5f3d581ce75b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:48.641032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2673b14-4970-469b-bc01-5f3d581ce75b None admin] PATH_INFO: `/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/2de0ff99a2b74979a28a8b10e9ebdfcd/roles/624a5f7ed1cf440cb177291e89bb11ff` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:48.649306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744684c-21b2-4b99-a524-c88677bf2ee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:48.654743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-1744684c-21b2-4b99-a524-c88677bf2ee9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:48.654743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:48.654743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:48.654743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:48.654743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:48.654743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:48.654743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:48.654743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:48.654743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:48.654743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:48.654743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:48.654743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:48.654743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:48.654743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:48.654743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:48.654743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:48.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:48.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:48.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:48.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:48.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:48.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:48.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:48.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:48.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:48.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:48.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:48.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:48.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:48.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:48.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:48.655704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:48.656560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:48.656560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:48.656560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:48.656560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:48.656560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:48.656560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:48.656560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:48.656560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:48.656560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:48.656560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:48.656560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:48.656560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:48.656560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 299/583] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:48.658033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744684c-21b2-4b99-a524-c88677bf2ee9 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:48.661693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2673b14-4970-469b-bc01-5f3d581ce75b None admin] RBAC: Authorizing `identity:create_grant(project_id=218a9ee5cb2842248b6995d2562bb74b, user_id=2de0ff99a2b74979a28a8b10e9ebdfcd, role_id=624a5f7ed1cf440cb177291e89bb11ff)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:48.663195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2673b14-4970-469b-bc01-5f3d581ce75b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:48.679062 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 285/584] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:10:48 2022] PUT /identity/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/2de0ff99a2b74979a28a8b10e9ebdfcd/roles/624a5f7ed1cf440cb177291e89bb11ff => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:48.680281 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2673b14-4970-469b-bc01-5f3d581ce75b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:48.756281 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:48.790952 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:48.813698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744684c-21b2-4b99-a524-c88677bf2ee9 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:48.818037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2673b14-4970-469b-bc01-5f3d581ce75b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:48.853067 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6279904-b61a-4e00-8717-11709b9982d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:48.854084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6279904-b61a-4e00-8717-11709b9982d0 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:48.854738 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6279904-b61a-4e00-8717-11709b9982d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:48.855220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6279904-b61a-4e00-8717-11709b9982d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:48.856077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6279904-b61a-4e00-8717-11709b9982d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:48.856536 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6279904-b61a-4e00-8717-11709b9982d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:48.858148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1742c70-609f-418f-baa2-163cc286d64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:48.858610 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6279904-b61a-4e00-8717-11709b9982d0 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:48.859022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1742c70-609f-418f-baa2-163cc286d64a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:48.859392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1742c70-609f-418f-baa2-163cc286d64a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:48.859807 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1742c70-609f-418f-baa2-163cc286d64a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:48.871166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1742c70-609f-418f-baa2-163cc286d64a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:48.872644 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1742c70-609f-418f-baa2-163cc286d64a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:48.873471 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 300/585] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:48.874572 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6279904-b61a-4e00-8717-11709b9982d0 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:48.898396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-b1742c70-609f-418f-baa2-163cc286d64a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:48.898396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:48.898396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:48.898396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:48.898396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:48.898396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:48.898396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:48.898396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:48.898396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:48.898396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:48.898396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:48.898396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:48.898396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:48.898396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:48.898396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:48.899409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:48.900477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:48.900477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:48.900477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:48.900477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:48.906994 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 286/586] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:48.908193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1742c70-609f-418f-baa2-163cc286d64a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:49.009315 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6279904-b61a-4e00-8717-11709b9982d0 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:49.030309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1742c70-609f-418f-baa2-163cc286d64a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:49.039575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:49.050950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00382b2-e68a-40cc-a229-a80854bdb0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:49.051791 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00382b2-e68a-40cc-a229-a80854bdb0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:49.052271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00382b2-e68a-40cc-a229-a80854bdb0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:49.052777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00382b2-e68a-40cc-a229-a80854bdb0f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:49.053805 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00382b2-e68a-40cc-a229-a80854bdb0f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:49.054190 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00382b2-e68a-40cc-a229-a80854bdb0f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:49.055656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00382b2-e68a-40cc-a229-a80854bdb0f1 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:49.069268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 301/587] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:10:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:49.070488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00382b2-e68a-40cc-a229-a80854bdb0f1 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:49.072857 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:49.080281 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b3768-8832-4a3f-a087-027053424bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:49.081408 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b3768-8832-4a3f-a087-027053424bee None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:49.081951 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b3768-8832-4a3f-a087-027053424bee None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:49.082434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b3768-8832-4a3f-a087-027053424bee None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:49.091431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b3768-8832-4a3f-a087-027053424bee None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:49.093200 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b3768-8832-4a3f-a087-027053424bee None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:49.106781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-fd7b3768-8832-4a3f-a087-027053424bee None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:10:49.106781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:49.106781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:49.106781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:49.106781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:49.106781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:49.106781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:49.106781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:49.106781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:49.106781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:49.106781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:49.106781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:49.106781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:10:49.106781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:10:49.106781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:10:49.106781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:49.108175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:49.109177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:49.109177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:49.109177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:10:49.109177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:10:49.109177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:10:49.109177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:10:49.109177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:10:49.109177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:10:49.109177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:10:49.109177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:49.111856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 287/588] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:49.117249 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b3768-8832-4a3f-a087-027053424bee None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:49.175967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00382b2-e68a-40cc-a229-a80854bdb0f1 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:49.212611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58125ced-e961-4aee-a198-435734d15eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:49.213249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58125ced-e961-4aee-a198-435734d15eab None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:49.213529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58125ced-e961-4aee-a198-435734d15eab None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:49.213821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58125ced-e961-4aee-a198-435734d15eab None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:49.214634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58125ced-e961-4aee-a198-435734d15eab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:49.214944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58125ced-e961-4aee-a198-435734d15eab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:49.216380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58125ced-e961-4aee-a198-435734d15eab None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:49.228837 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b3768-8832-4a3f-a087-027053424bee None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:49.229996 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 302/589] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:49.230988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58125ced-e961-4aee-a198-435734d15eab None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:49.266726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634caf6f-ab51-47d7-aa4b-4575e049ac34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:49.267903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634caf6f-ab51-47d7-aa4b-4575e049ac34 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:49.268387 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634caf6f-ab51-47d7-aa4b-4575e049ac34 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:49.268882 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634caf6f-ab51-47d7-aa4b-4575e049ac34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:49.281875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634caf6f-ab51-47d7-aa4b-4575e049ac34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:49.283660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:49.284987 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634caf6f-ab51-47d7-aa4b-4575e049ac34 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:49.302702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-634caf6f-ab51-47d7-aa4b-4575e049ac34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:49.302702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:49.302702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:49.302702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:49.302702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:49.302702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:49.302702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:49.302702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:49.302702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:49.302702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:49.302702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:49.302702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:49.302702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:49.302702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:49.302702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:49.303745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:49.304634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:49.304634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:49.304634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:49.304634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:49.307058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 288/590] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:49.308069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634caf6f-ab51-47d7-aa4b-4575e049ac34 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:49.350088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58125ced-e961-4aee-a198-435734d15eab None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:49.375376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:49.395745 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d491aa9-b699-4585-87d3-25cced948851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:49.396436 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d491aa9-b699-4585-87d3-25cced948851 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:49.396725 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d491aa9-b699-4585-87d3-25cced948851 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:49.396994 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d491aa9-b699-4585-87d3-25cced948851 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:49.397862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d491aa9-b699-4585-87d3-25cced948851 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:49.398223 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d491aa9-b699-4585-87d3-25cced948851 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:49.399962 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d491aa9-b699-4585-87d3-25cced948851 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:49.426605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 303/591] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 15:10:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:49.427474 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d491aa9-b699-4585-87d3-25cced948851 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:49.428895 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634caf6f-ab51-47d7-aa4b-4575e049ac34 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:49.468537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744d02b-2768-4a1e-ab08-d00a182eb546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:49.469403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744d02b-2768-4a1e-ab08-d00a182eb546 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:49.469837 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744d02b-2768-4a1e-ab08-d00a182eb546 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:49.470209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744d02b-2768-4a1e-ab08-d00a182eb546 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:49.477988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f744d02b-2768-4a1e-ab08-d00a182eb546 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:49.479728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f744d02b-2768-4a1e-ab08-d00a182eb546 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:49.496108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-f744d02b-2768-4a1e-ab08-d00a182eb546 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:49.496108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:49.496108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:49.496108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:49.496108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:49.496108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:49.496108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:49.496108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:49.496108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:49.496108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:49.496108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:49.496108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:49.496108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:10:49.496108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:10:49.496108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:49.497284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:49.498551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:10:49.498551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:10:49.498551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:10:49.498551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:10:49.498551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:10:49.498551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:10:49.498551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:49.498551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:49.501809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 289/592] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:49.501809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744d02b-2768-4a1e-ab08-d00a182eb546 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:49.567590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d491aa9-b699-4585-87d3-25cced948851 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:49.580481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:49.608532 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d232e5-a552-4d2f-95da-55b451ad7639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:49.609516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d232e5-a552-4d2f-95da-55b451ad7639 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:49.610016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d232e5-a552-4d2f-95da-55b451ad7639 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:49.610568 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d232e5-a552-4d2f-95da-55b451ad7639 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:49.611442 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d232e5-a552-4d2f-95da-55b451ad7639 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:49.611931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d232e5-a552-4d2f-95da-55b451ad7639 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:49.613501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d232e5-a552-4d2f-95da-55b451ad7639 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:49.613866 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744d02b-2768-4a1e-ab08-d00a182eb546 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:49.615561 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:49.631442 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 304/593] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:49.632606 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d232e5-a552-4d2f-95da-55b451ad7639 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:49.657689 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551b652-92d8-4b1d-b54f-0112b9551434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:49.658389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551b652-92d8-4b1d-b54f-0112b9551434 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:49.658804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551b652-92d8-4b1d-b54f-0112b9551434 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:49.659086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551b652-92d8-4b1d-b54f-0112b9551434 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:49.673974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551b652-92d8-4b1d-b54f-0112b9551434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:49.675449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551b652-92d8-4b1d-b54f-0112b9551434 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:49.696107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-a551b652-92d8-4b1d-b54f-0112b9551434 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:49.696107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:49.696107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:49.696107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:49.696107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:49.696107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:49.696107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:49.696107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:49.696107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:49.696107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:49.696107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:49.696107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:49.696107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:49.696107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:49.696107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:49.697018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:49.697957 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:49.697957 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:49.697957 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:49.697957 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:49.700147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 290/594] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:49.701194 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551b652-92d8-4b1d-b54f-0112b9551434 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:49.763487 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d232e5-a552-4d2f-95da-55b451ad7639 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:49.804663 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9bced-807d-4d67-ae62-fffbdfd09c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:49.805711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9bced-807d-4d67-ae62-fffbdfd09c3d None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:49.806276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9bced-807d-4d67-ae62-fffbdfd09c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:49.806902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9bced-807d-4d67-ae62-fffbdfd09c3d None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:49.807881 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9bced-807d-4d67-ae62-fffbdfd09c3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:49.808404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9bced-807d-4d67-ae62-fffbdfd09c3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:49.810086 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9bced-807d-4d67-ae62-fffbdfd09c3d None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:49.816865 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551b652-92d8-4b1d-b54f-0112b9551434 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:49.824918 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 305/595] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:10:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:49.826189 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9bced-807d-4d67-ae62-fffbdfd09c3d None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:49.862572 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:49.870010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7603c1-56b1-43a4-84c1-885b4593ac33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:49.870689 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7603c1-56b1-43a4-84c1-885b4593ac33 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:49.870968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7603c1-56b1-43a4-84c1-885b4593ac33 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:49.871246 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7603c1-56b1-43a4-84c1-885b4593ac33 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:49.885811 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7603c1-56b1-43a4-84c1-885b4593ac33 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:49.887331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7603c1-56b1-43a4-84c1-885b4593ac33 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:49.900318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-fc7603c1-56b1-43a4-84c1-885b4593ac33 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:10:49.900318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:49.900318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:49.900318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:49.900318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:49.900318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:49.900318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:49.900318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:49.900318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:49.900318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:49.900318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:49.900318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:49.900318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:10:49.900318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:10:49.900318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:10:49.900318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:49.901343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:49.902991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:49.902991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:49.902991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:10:49.902991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:10:49.902991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:10:49.902991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:10:49.902991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:10:49.902991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:10:49.902991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:10:49.902991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:49.902991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 291/596] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:49.906674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7603c1-56b1-43a4-84c1-885b4593ac33 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:49.924228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:49.954582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9bced-807d-4d67-ae62-fffbdfd09c3d None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:49.991254 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573130bb-7ca1-4928-bed9-bd06cd9abd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:49.992306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573130bb-7ca1-4928-bed9-bd06cd9abd6e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:49.992784 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573130bb-7ca1-4928-bed9-bd06cd9abd6e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:49.993254 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573130bb-7ca1-4928-bed9-bd06cd9abd6e None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:49.994201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573130bb-7ca1-4928-bed9-bd06cd9abd6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:49.994708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573130bb-7ca1-4928-bed9-bd06cd9abd6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:49.996502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573130bb-7ca1-4928-bed9-bd06cd9abd6e None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:50.009592 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 306/597] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:50.010874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573130bb-7ca1-4928-bed9-bd06cd9abd6e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:50.017022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7603c1-56b1-43a4-84c1-885b4593ac33 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:50.052710 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add2d87-064d-4744-a098-cf01cc6856ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:50.053402 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add2d87-064d-4744-a098-cf01cc6856ae None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:50.053684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add2d87-064d-4744-a098-cf01cc6856ae None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:50.053986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add2d87-064d-4744-a098-cf01cc6856ae None admin] PATH_INFO: `/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/94f696dfe4b34c4aa19104b5f7ee6e70/roles/624a5f7ed1cf440cb177291e89bb11ff` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:50.069193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add2d87-064d-4744-a098-cf01cc6856ae None admin] RBAC: Authorizing `identity:create_grant(project_id=218a9ee5cb2842248b6995d2562bb74b, user_id=94f696dfe4b34c4aa19104b5f7ee6e70, role_id=624a5f7ed1cf440cb177291e89bb11ff)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:50.070682 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add2d87-064d-4744-a098-cf01cc6856ae None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:50.089894 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 292/598] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:10:49 2022] PUT /identity/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/94f696dfe4b34c4aa19104b5f7ee6e70/roles/624a5f7ed1cf440cb177291e89bb11ff => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:50.089894 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add2d87-064d-4744-a098-cf01cc6856ae None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:50.134310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:50.155354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573130bb-7ca1-4928-bed9-bd06cd9abd6e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:50.165877 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:50.213616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add2d87-064d-4744-a098-cf01cc6856ae None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:50.236030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d445c3-9bc5-47ec-8258-3af6b3c135f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:50.236689 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d445c3-9bc5-47ec-8258-3af6b3c135f2 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:50.236967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d445c3-9bc5-47ec-8258-3af6b3c135f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:50.237243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d445c3-9bc5-47ec-8258-3af6b3c135f2 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:50.238109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d445c3-9bc5-47ec-8258-3af6b3c135f2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:50.241227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d445c3-9bc5-47ec-8258-3af6b3c135f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:50.242833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d445c3-9bc5-47ec-8258-3af6b3c135f2 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:50.263517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d176df-2af2-4a22-a8b6-c469e4e467e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:50.264213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d176df-2af2-4a22-a8b6-c469e4e467e6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:50.264561 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d176df-2af2-4a22-a8b6-c469e4e467e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:50.264899 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d176df-2af2-4a22-a8b6-c469e4e467e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:50.277808 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d176df-2af2-4a22-a8b6-c469e4e467e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:50.277808 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d176df-2af2-4a22-a8b6-c469e4e467e6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:50.284432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 307/599] 10.222.0.48 () {56 vars in 1139 bytes} [Mon Aug 1 15:10:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:50.291557 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d445c3-9bc5-47ec-8258-3af6b3c135f2 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:50.293451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-25d176df-2af2-4a22-a8b6-c469e4e467e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:50.293451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:50.293451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:50.293451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:50.293451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:50.293451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:50.293451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:50.293451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:50.293451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:50.293451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:50.293451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:50.293451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:50.293451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:10:50.293451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:10:50.293451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:50.294336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:50.295652 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:10:50.295652 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:10:50.295652 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:10:50.295652 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:10:50.295652 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:10:50.295652 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:10:50.295652 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:50.295652 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:50.295652 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 293/600] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:50.301256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d176df-2af2-4a22-a8b6-c469e4e467e6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:50.419206 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d176df-2af2-4a22-a8b6-c469e4e467e6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:50.450555 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:50.461923 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431ffc7-6d59-4fc9-b1a6-8e8d15067194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:50.462922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431ffc7-6d59-4fc9-b1a6-8e8d15067194 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:50.463392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431ffc7-6d59-4fc9-b1a6-8e8d15067194 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:50.463865 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431ffc7-6d59-4fc9-b1a6-8e8d15067194 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:50.464289 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d445c3-9bc5-47ec-8258-3af6b3c135f2 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:50.465078 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431ffc7-6d59-4fc9-b1a6-8e8d15067194 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:50.465582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431ffc7-6d59-4fc9-b1a6-8e8d15067194 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:50.467259 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431ffc7-6d59-4fc9-b1a6-8e8d15067194 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:50.480059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 294/601] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:10:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:50.482197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1f116-bfd3-4224-8ddb-ef848c24bf81 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:50.483055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1f116-bfd3-4224-8ddb-ef848c24bf81 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:50.483533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1f116-bfd3-4224-8ddb-ef848c24bf81 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:50.485688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 295/602] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:50.486684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1f116-bfd3-4224-8ddb-ef848c24bf81 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:50.501409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756ba3f-8ad6-4cf1-a782-4b9413239bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:50.502085 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756ba3f-8ad6-4cf1-a782-4b9413239bea None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:50.502365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756ba3f-8ad6-4cf1-a782-4b9413239bea None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:50.502642 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756ba3f-8ad6-4cf1-a782-4b9413239bea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:50.513607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756ba3f-8ad6-4cf1-a782-4b9413239bea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:50.515106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756ba3f-8ad6-4cf1-a782-4b9413239bea None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:50.523489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:50.533090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-1756ba3f-8ad6-4cf1-a782-4b9413239bea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:50.533090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:50.533090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:50.533090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:50.533090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:50.533090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:50.533090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:50.533090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:50.533090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:50.533090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:50.533090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:50.533090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:50.533090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:50.533090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:50.533090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:50.534049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:50.534949 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:50.534949 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:50.534949 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:50.534949 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:50.535736 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 308/603] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:50.541143 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced6736-06b9-49ef-b0fd-f49975fe5705 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:50.541474 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced6736-06b9-49ef-b0fd-f49975fe5705 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:50.541760 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced6736-06b9-49ef-b0fd-f49975fe5705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:50.560902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2ced6736-06b9-49ef-b0fd-f49975fe5705 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:50.562171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2ced6736-06b9-49ef-b0fd-f49975fe5705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:50.603316 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1f116-bfd3-4224-8ddb-ef848c24bf81 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:50.638791 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615fc61-27fd-42ea-a337-d66376dd013c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:50.639151 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 309/604] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:50.639985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced6736-06b9-49ef-b0fd-f49975fe5705 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:50.640373 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615fc61-27fd-42ea-a337-d66376dd013c None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:50.640839 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615fc61-27fd-42ea-a337-d66376dd013c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:50.641325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615fc61-27fd-42ea-a337-d66376dd013c None admin] PATH_INFO: `/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/63824ba9369c4899a4b946588cbdae24/roles/624a5f7ed1cf440cb177291e89bb11ff` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:50.656785 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1615fc61-27fd-42ea-a337-d66376dd013c None admin] RBAC: Authorizing `identity:create_grant(project_id=218a9ee5cb2842248b6995d2562bb74b, user_id=63824ba9369c4899a4b946588cbdae24, role_id=624a5f7ed1cf440cb177291e89bb11ff)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:50.658749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1615fc61-27fd-42ea-a337-d66376dd013c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:50.678298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 296/605] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:10:50 2022] PUT /identity/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/63824ba9369c4899a4b946588cbdae24/roles/624a5f7ed1cf440cb177291e89bb11ff => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:50.680840 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615fc61-27fd-42ea-a337-d66376dd013c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:50.746219 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:50.795294 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced6736-06b9-49ef-b0fd-f49975fe5705 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:50.829841 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:50.841192 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f76dc-98a3-4335-8482-3790a84f9a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:50.846035 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f76dc-98a3-4335-8482-3790a84f9a19 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:50.846383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f76dc-98a3-4335-8482-3790a84f9a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:50.846653 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f76dc-98a3-4335-8482-3790a84f9a19 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:50.847410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f76dc-98a3-4335-8482-3790a84f9a19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:50.847711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f76dc-98a3-4335-8482-3790a84f9a19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:50.849131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f76dc-98a3-4335-8482-3790a84f9a19 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:50.875882 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615fc61-27fd-42ea-a337-d66376dd013c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:50.878446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 310/606] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:50.880399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd473bc-e185-40fc-86ca-c24e77d107f4 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:50.880399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd473bc-e185-40fc-86ca-c24e77d107f4 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:50.880721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd473bc-e185-40fc-86ca-c24e77d107f4 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:50.882731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 311/607] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:50.888017 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd473bc-e185-40fc-86ca-c24e77d107f4 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:50.924505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e1c8b-2a92-4b6c-9edd-27a4e14a1896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:50.925159 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e1c8b-2a92-4b6c-9edd-27a4e14a1896 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:50.925430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e1c8b-2a92-4b6c-9edd-27a4e14a1896 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:50.925699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e1c8b-2a92-4b6c-9edd-27a4e14a1896 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:50.938417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e1c8b-2a92-4b6c-9edd-27a4e14a1896 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:50.944812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-535e1c8b-2a92-4b6c-9edd-27a4e14a1896 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:50.944812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:50.944812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:50.944812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:50.944812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:50.944812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:50.944812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:50.944812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:50.944812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:50.944812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:50.944812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:50.944812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:50.944812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:50.944812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:50.944812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:50.944812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:50.945802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:50.945802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:50.945802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:50.945802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:50.945802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:50.945802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:50.945802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:50.945802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:50.945802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:50.945802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:50.945802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:50.945802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:50.945802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:50.945802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:50.945802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:50.945802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:50.947063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:50.947063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:50.947063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:50.947063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:50.947063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:50.947063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:50.947063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:50.947063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:50.947063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:50.947063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:50.947063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:50.947063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:50.947063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 297/608] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:50.947801 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a723ffe-c8f4-47f9-be43-9639562499fe None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:50.947801 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a723ffe-c8f4-47f9-be43-9639562499fe None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:50.948195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a723ffe-c8f4-47f9-be43-9639562499fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:50.974064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-9a723ffe-c8f4-47f9-be43-9639562499fe None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:50.975776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9a723ffe-c8f4-47f9-be43-9639562499fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:51.011519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd473bc-e185-40fc-86ca-c24e77d107f4 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:51.055152 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f984e-8455-4728-86a2-95d49acefd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:51.056055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f984e-8455-4728-86a2-95d49acefd03 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:51.056391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f984e-8455-4728-86a2-95d49acefd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:51.057080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f984e-8455-4728-86a2-95d49acefd03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:51.061504 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 298/609] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:51.062486 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a723ffe-c8f4-47f9-be43-9639562499fe None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:51.073877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f984e-8455-4728-86a2-95d49acefd03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:51.077934 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f984e-8455-4728-86a2-95d49acefd03 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:51.096302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-b96f984e-8455-4728-86a2-95d49acefd03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:51.096302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:51.096302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:51.096302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:51.096302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:51.096302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:51.096302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:51.096302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:51.096302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:51.096302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:51.096302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:51.096302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:51.096302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:10:51.096302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:10:51.096302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:51.098501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:51.100741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:10:51.100741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:10:51.100741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:10:51.100741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:10:51.100741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:10:51.100741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:10:51.100741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:51.100741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:51.105886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 312/610] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:51.105886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f984e-8455-4728-86a2-95d49acefd03 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:51.107282 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:51.159689 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:51.200106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a723ffe-c8f4-47f9-be43-9639562499fe None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:51.226991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f984e-8455-4728-86a2-95d49acefd03 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:51.238035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69795964-c8e9-43b4-b4bc-30e770792bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:51.238797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69795964-c8e9-43b4-b4bc-30e770792bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:51.239179 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69795964-c8e9-43b4-b4bc-30e770792bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:51.239513 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69795964-c8e9-43b4-b4bc-30e770792bd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:51.240488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69795964-c8e9-43b4-b4bc-30e770792bd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:51.240855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69795964-c8e9-43b4-b4bc-30e770792bd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:51.242371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69795964-c8e9-43b4-b4bc-30e770792bd7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:51.256454 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 299/611] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:10:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:51.257570 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69795964-c8e9-43b4-b4bc-30e770792bd7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:51.272068 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a66043-a88a-4e32-a87d-8bdea0be37bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:51.273352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a66043-a88a-4e32-a87d-8bdea0be37bb None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:51.273636 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a66043-a88a-4e32-a87d-8bdea0be37bb None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:51.273996 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a66043-a88a-4e32-a87d-8bdea0be37bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:51.287706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a66043-a88a-4e32-a87d-8bdea0be37bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:51.295029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-75a66043-a88a-4e32-a87d-8bdea0be37bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:51.295029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:51.295029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:51.295029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:51.295029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:51.295029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:51.295029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:51.295029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:51.295029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:51.295029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:51.295029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:51.295029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:51.295029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:51.295029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:51.295029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:51.295029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:51.296514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:51.296514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:51.296514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:51.296514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:51.296514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:51.296514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:51.296514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:51.296514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:51.296514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:51.296514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:51.296514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:51.296514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:51.296514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:51.296514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:51.296514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:51.296514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:51.297862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:51.297862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:51.297862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:51.297862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:51.297862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:51.297862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:51.297862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:51.297862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:51.297862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:51.297862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:51.297862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:51.297862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:51.305800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 313/612] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:51.305800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a66043-a88a-4e32-a87d-8bdea0be37bb None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:51.364320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:51.383121 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69795964-c8e9-43b4-b4bc-30e770792bd7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:51.418684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72950d8b-0a5e-4149-bc5a-8ca70fc026e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:51.419734 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72950d8b-0a5e-4149-bc5a-8ca70fc026e6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:51.420221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72950d8b-0a5e-4149-bc5a-8ca70fc026e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:51.420688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72950d8b-0a5e-4149-bc5a-8ca70fc026e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:51.421649 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72950d8b-0a5e-4149-bc5a-8ca70fc026e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:51.422170 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72950d8b-0a5e-4149-bc5a-8ca70fc026e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:51.423034 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a66043-a88a-4e32-a87d-8bdea0be37bb None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:51.424078 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72950d8b-0a5e-4149-bc5a-8ca70fc026e6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:51.432895 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:51.442863 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 300/613] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:10:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:51.444087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72950d8b-0a5e-4149-bc5a-8ca70fc026e6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:51.461813 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769f395-a32b-4c39-8d73-ad98e6facbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:51.462404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769f395-a32b-4c39-8d73-ad98e6facbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:51.462687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769f395-a32b-4c39-8d73-ad98e6facbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:51.462963 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769f395-a32b-4c39-8d73-ad98e6facbe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:51.475241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769f395-a32b-4c39-8d73-ad98e6facbe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:51.476840 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769f395-a32b-4c39-8d73-ad98e6facbe8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:51.495446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-5769f395-a32b-4c39-8d73-ad98e6facbe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:51.495446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:51.495446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:51.495446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:51.495446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:51.495446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:51.495446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:51.495446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:51.495446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:51.495446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:51.495446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:51.495446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:51.495446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:51.495446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:51.495446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:51.496429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:51.497335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:51.497335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:51.497335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:51.497335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:51.497335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 314/614] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:51.501950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769f395-a32b-4c39-8d73-ad98e6facbe8 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:51.556935 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72950d8b-0a5e-4149-bc5a-8ca70fc026e6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:51.595966 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9be91c-2ab6-4700-92e9-846ae0ceaf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:51.597017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9be91c-2ab6-4700-92e9-846ae0ceaf05 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:51.597492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9be91c-2ab6-4700-92e9-846ae0ceaf05 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:51.598025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9be91c-2ab6-4700-92e9-846ae0ceaf05 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:51.598943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9be91c-2ab6-4700-92e9-846ae0ceaf05 None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:51.599507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9be91c-2ab6-4700-92e9-846ae0ceaf05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:51.601128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9be91c-2ab6-4700-92e9-846ae0ceaf05 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:51.610904 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 301/615] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:10:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:51.612086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9be91c-2ab6-4700-92e9-846ae0ceaf05 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:51.617280 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769f395-a32b-4c39-8d73-ad98e6facbe8 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:51.624046 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:51.658517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15814d6f-2ef8-409a-88f5-2c196dceece6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:51.659481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15814d6f-2ef8-409a-88f5-2c196dceece6 None admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:51.659906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15814d6f-2ef8-409a-88f5-2c196dceece6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:51.660406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15814d6f-2ef8-409a-88f5-2c196dceece6 None admin] PATH_INFO: `/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/5ef6577b10f7426a925d594e627f27cf/roles/624a5f7ed1cf440cb177291e89bb11ff` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:51.676644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15814d6f-2ef8-409a-88f5-2c196dceece6 None admin] RBAC: Authorizing `identity:create_grant(project_id=218a9ee5cb2842248b6995d2562bb74b, user_id=5ef6577b10f7426a925d594e627f27cf, role_id=624a5f7ed1cf440cb177291e89bb11ff)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:51.678651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15814d6f-2ef8-409a-88f5-2c196dceece6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:51.695053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 315/616] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:10:51 2022] PUT /identity/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/5ef6577b10f7426a925d594e627f27cf/roles/624a5f7ed1cf440cb177291e89bb11ff => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:51.696712 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15814d6f-2ef8-409a-88f5-2c196dceece6 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:51.718728 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:51.748281 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9be91c-2ab6-4700-92e9-846ae0ceaf05 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:51.837515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0370818-947a-4209-93bb-9f51e48d9fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:51.840421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0370818-947a-4209-93bb-9f51e48d9fdb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:51.842219 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0370818-947a-4209-93bb-9f51e48d9fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:51.842507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0370818-947a-4209-93bb-9f51e48d9fdb None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:51.843234 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0370818-947a-4209-93bb-9f51e48d9fdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:51.843539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0370818-947a-4209-93bb-9f51e48d9fdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:51.845122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0370818-947a-4209-93bb-9f51e48d9fdb None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:51.876371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 302/617] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:51.883272 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0370818-947a-4209-93bb-9f51e48d9fdb None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:51.914289 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15814d6f-2ef8-409a-88f5-2c196dceece6 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:51.961139 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad811f-3239-4132-97ed-1d5c9fc21364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:51.961810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad811f-3239-4132-97ed-1d5c9fc21364 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:51.962120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad811f-3239-4132-97ed-1d5c9fc21364 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:51.962455 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad811f-3239-4132-97ed-1d5c9fc21364 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:51.979364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:51.979563 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad811f-3239-4132-97ed-1d5c9fc21364 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:51.980497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad811f-3239-4132-97ed-1d5c9fc21364 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:52.003010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-38ad811f-3239-4132-97ed-1d5c9fc21364 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:52.003010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:52.003010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:52.003010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:52.003010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:52.003010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:52.003010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:52.003010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:52.003010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:52.003010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:52.003010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:52.003010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:52.003010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:52.003010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:52.003010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:52.003988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:52.004896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:52.004896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:52.004896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:52.004896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:52.004896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 316/618] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:52.012525 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad811f-3239-4132-97ed-1d5c9fc21364 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:52.037269 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0370818-947a-4209-93bb-9f51e48d9fdb None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:52.100174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5816f-5f7e-48c4-91c2-85b690c412a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:52.101174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5816f-5f7e-48c4-91c2-85b690c412a2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:52.101632 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5816f-5f7e-48c4-91c2-85b690c412a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:52.102094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5816f-5f7e-48c4-91c2-85b690c412a2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:52.114319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5816f-5f7e-48c4-91c2-85b690c412a2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:52.117199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5816f-5f7e-48c4-91c2-85b690c412a2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:52.130762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:52.136439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-75a5816f-5f7e-48c4-91c2-85b690c412a2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:10:52.136439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:52.136439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:52.136439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:52.136439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:52.136439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:52.136439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:52.136439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:52.136439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:52.136439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:52.136439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:52.136439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:52.136439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:10:52.136439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:10:52.136439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:10:52.136439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:52.137505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:52.138626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:52.138626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:52.138626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:10:52.138626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:10:52.138626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:10:52.138626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:10:52.138626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:10:52.138626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:10:52.138626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:10:52.138626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:52.146247 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 303/619] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:10:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:52.152424 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5816f-5f7e-48c4-91c2-85b690c412a2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:52.203001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad811f-3239-4132-97ed-1d5c9fc21364 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:52.267007 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b69da-3202-4f9a-a314-d30ec5e480d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:52.267928 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b69da-3202-4f9a-a314-d30ec5e480d7 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:52.268353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b69da-3202-4f9a-a314-d30ec5e480d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:52.268798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b69da-3202-4f9a-a314-d30ec5e480d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:52.269665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b69da-3202-4f9a-a314-d30ec5e480d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:52.270144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b69da-3202-4f9a-a314-d30ec5e480d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:52.271692 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b69da-3202-4f9a-a314-d30ec5e480d7 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:52.299707 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 317/620] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:52.301523 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b69da-3202-4f9a-a314-d30ec5e480d7 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:52.320354 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5816f-5f7e-48c4-91c2-85b690c412a2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:52.349210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:52.362790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a670e9-64b3-405d-863a-6cea89531c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:52.363848 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a670e9-64b3-405d-863a-6cea89531c72 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:52.364304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a670e9-64b3-405d-863a-6cea89531c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:52.364725 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a670e9-64b3-405d-863a-6cea89531c72 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:52.365682 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a670e9-64b3-405d-863a-6cea89531c72 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:52.366183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a670e9-64b3-405d-863a-6cea89531c72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:52.368492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a670e9-64b3-405d-863a-6cea89531c72 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:52.389562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 304/621] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 15:10:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:52.395111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a670e9-64b3-405d-863a-6cea89531c72 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:52.519942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a670e9-64b3-405d-863a-6cea89531c72 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:52.543829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b69da-3202-4f9a-a314-d30ec5e480d7 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:52.559658 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56305c3-4bde-48be-9458-b27153eb5343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:52.560260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56305c3-4bde-48be-9458-b27153eb5343 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:52.560539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56305c3-4bde-48be-9458-b27153eb5343 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:52.560807 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56305c3-4bde-48be-9458-b27153eb5343 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:52.571425 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56305c3-4bde-48be-9458-b27153eb5343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:52.572949 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56305c3-4bde-48be-9458-b27153eb5343 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:52.574224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:52.588986 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f62f57-787e-4eb8-9b82-21048e8be241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:52.589712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e56305c3-4bde-48be-9458-b27153eb5343 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:52.589712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:52.589712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:52.589712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:52.589712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:52.589712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:52.589712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:52.589712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:52.589712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:52.589712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:52.589712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:52.589712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:52.589712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:52.589712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:52.589712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:52.591059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:52.592005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:52.592005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:52.592005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:52.592005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:52.592931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f62f57-787e-4eb8-9b82-21048e8be241 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:52.593340 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f62f57-787e-4eb8-9b82-21048e8be241 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:52.593748 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f62f57-787e-4eb8-9b82-21048e8be241 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:52.597407 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 305/622] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:52.598842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebd29a-4ff1-4f70-a233-3e5e3cd6a523 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:52.599308 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebd29a-4ff1-4f70-a233-3e5e3cd6a523 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:52.599452 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ebd29a-4ff1-4f70-a233-3e5e3cd6a523 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:52.600969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 306/623] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:52.601654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebd29a-4ff1-4f70-a233-3e5e3cd6a523 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:52.606388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f62f57-787e-4eb8-9b82-21048e8be241 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:52.608510 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f62f57-787e-4eb8-9b82-21048e8be241 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:52.624696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-38f62f57-787e-4eb8-9b82-21048e8be241 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:10:52.624696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:52.624696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:52.624696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:52.624696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:52.624696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:52.624696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:52.624696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:52.624696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:52.624696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:52.624696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:52.624696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:52.624696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:10:52.624696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:10:52.624696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:10:52.624696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:52.626167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:52.627133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:52.627133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:52.627133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:10:52.627133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:10:52.627133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:10:52.627133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:10:52.627133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:10:52.627133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:10:52.627133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:10:52.627133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:52.627685 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 318/624] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 15:10:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:52.633290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c922011-4117-4f77-929e-68cb39d735a5 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:52.633290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c922011-4117-4f77-929e-68cb39d735a5 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:52.633290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c922011-4117-4f77-929e-68cb39d735a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:52.653037 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-7c922011-4117-4f77-929e-68cb39d735a5 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:52.654659 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-7c922011-4117-4f77-929e-68cb39d735a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:52.656686 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:52.723851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ebd29a-4ff1-4f70-a233-3e5e3cd6a523 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:52.748760 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 319/625] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:52.750043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c922011-4117-4f77-929e-68cb39d735a5 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:52.786657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85189192-30d2-4a23-a45a-79d154f3179b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:52.787276 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85189192-30d2-4a23-a45a-79d154f3179b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:52.787556 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85189192-30d2-4a23-a45a-79d154f3179b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:52.787824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85189192-30d2-4a23-a45a-79d154f3179b None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:52.788520 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85189192-30d2-4a23-a45a-79d154f3179b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:52.788817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85189192-30d2-4a23-a45a-79d154f3179b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:52.790254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85189192-30d2-4a23-a45a-79d154f3179b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:52.807805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 307/626] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:52.808774 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85189192-30d2-4a23-a45a-79d154f3179b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:52.870336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c922011-4117-4f77-929e-68cb39d735a5 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:52.886035 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:52.907831 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6b1ba-49cd-4140-96b4-44d3c6126219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:52.908464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6b1ba-49cd-4140-96b4-44d3c6126219 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:52.908746 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6b1ba-49cd-4140-96b4-44d3c6126219 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:52.909019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6b1ba-49cd-4140-96b4-44d3c6126219 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:52.909865 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6b1ba-49cd-4140-96b4-44d3c6126219 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:52.910175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6b1ba-49cd-4140-96b4-44d3c6126219 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:52.911628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6b1ba-49cd-4140-96b4-44d3c6126219 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:52.917224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:52.921617 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85189192-30d2-4a23-a45a-79d154f3179b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:52.933483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 320/627] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 15:10:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:52.933483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6b1ba-49cd-4140-96b4-44d3c6126219 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:52.964949 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85585526-73c9-45ae-a940-7edbd48da8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:52.965604 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85585526-73c9-45ae-a940-7edbd48da8ef None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:52.966037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85585526-73c9-45ae-a940-7edbd48da8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:52.966307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85585526-73c9-45ae-a940-7edbd48da8ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:52.978137 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85585526-73c9-45ae-a940-7edbd48da8ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:52.983910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-85585526-73c9-45ae-a940-7edbd48da8ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:52.983910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:52.983910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:52.983910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:52.983910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:52.983910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:52.983910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:52.983910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:52.983910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:52.983910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:52.983910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:52.983910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:52.983910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:52.983910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:52.983910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:52.983910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:52.984842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:52.984842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:52.984842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:52.984842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:52.984842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:52.984842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:52.984842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:52.984842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:52.984842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:52.984842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:52.984842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:52.984842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:52.984842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:52.984842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:52.984842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:52.984842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:52.985755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:52.985755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:52.985755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:52.985755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:52.985755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:52.985755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:52.985755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:52.985755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:52.985755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:52.985755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:52.985755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:52.985755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:52.985755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 308/628] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:52.987002 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849996ea-1457-4e56-981d-eddd8afc9961 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:52.987002 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849996ea-1457-4e56-981d-eddd8afc9961 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:52.987002 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849996ea-1457-4e56-981d-eddd8afc9961 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:52.988387 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 309/629] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:52.988974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849996ea-1457-4e56-981d-eddd8afc9961 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:53.048656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6b1ba-49cd-4140-96b4-44d3c6126219 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:53.088591 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efd538-c70b-4c5b-82e8-243f476793ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:53.089543 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efd538-c70b-4c5b-82e8-243f476793ab None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:53.089844 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efd538-c70b-4c5b-82e8-243f476793ab None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:53.090121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efd538-c70b-4c5b-82e8-243f476793ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:53.098142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efd538-c70b-4c5b-82e8-243f476793ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:53.099597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efd538-c70b-4c5b-82e8-243f476793ab None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:53.101137 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849996ea-1457-4e56-981d-eddd8afc9961 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:53.111580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-03efd538-c70b-4c5b-82e8-243f476793ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:53.111580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:53.111580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:53.111580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:53.111580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:53.111580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:53.111580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:53.111580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:53.111580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:53.111580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:53.111580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:53.111580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:53.111580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:10:53.111580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:10:53.111580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:53.112519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:53.113414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:10:53.113414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:10:53.113414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:10:53.113414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:10:53.113414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:10:53.113414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:10:53.113414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:53.113414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:53.113414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 321/630] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:53.113939 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efd538-c70b-4c5b-82e8-243f476793ab None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:53.141558 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fe6fd-b427-4d45-acdc-bd457136a453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:53.142220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fe6fd-b427-4d45-acdc-bd457136a453 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:53.142503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fe6fd-b427-4d45-acdc-bd457136a453 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:53.142781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fe6fd-b427-4d45-acdc-bd457136a453 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:53.144252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:53.170798 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:53.215332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0fe6fd-b427-4d45-acdc-bd457136a453 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:53.216941 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0fe6fd-b427-4d45-acdc-bd457136a453 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:53.219446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efd538-c70b-4c5b-82e8-243f476793ab None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:53.256305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d8b4c-b895-402d-b587-b3b5fafdab5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:53.256934 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d8b4c-b895-402d-b587-b3b5fafdab5b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:53.257213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d8b4c-b895-402d-b587-b3b5fafdab5b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:53.257487 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d8b4c-b895-402d-b587-b3b5fafdab5b None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:53.258253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d8b4c-b895-402d-b587-b3b5fafdab5b None admin] RBAC: Adding query filter params (name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:53.258559 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d8b4c-b895-402d-b587-b3b5fafdab5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:53.259960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d8b4c-b895-402d-b587-b3b5fafdab5b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:53.268337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 322/631] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:10:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:53.269646 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf19374-0a47-4f90-8170-2b02b6094119 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:53.269958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf19374-0a47-4f90-8170-2b02b6094119 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:53.270227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf19374-0a47-4f90-8170-2b02b6094119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:53.288274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-fdf19374-0a47-4f90-8170-2b02b6094119 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:53.289493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-fdf19374-0a47-4f90-8170-2b02b6094119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:53.328020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 310/632] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:53.329335 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fe6fd-b427-4d45-acdc-bd457136a453 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:53.365903 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 323/633] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:53.367430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf19374-0a47-4f90-8170-2b02b6094119 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:53.398006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:53.403933 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:53.446699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fe6fd-b427-4d45-acdc-bd457136a453 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:53.481044 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf19374-0a47-4f90-8170-2b02b6094119 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:53.483805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60e1e6-072a-4524-82d0-60ff740ddae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:53.484434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60e1e6-072a-4524-82d0-60ff740ddae8 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:53.484716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60e1e6-072a-4524-82d0-60ff740ddae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:53.485018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60e1e6-072a-4524-82d0-60ff740ddae8 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:53.485800 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60e1e6-072a-4524-82d0-60ff740ddae8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:53.486115 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60e1e6-072a-4524-82d0-60ff740ddae8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:53.487570 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60e1e6-072a-4524-82d0-60ff740ddae8 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:53.500031 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 311/634] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:10:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:53.501304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60e1e6-072a-4524-82d0-60ff740ddae8 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:53.520314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f652b7-aa59-47e5-8747-e74340d3f4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:53.521057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f652b7-aa59-47e5-8747-e74340d3f4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:53.521334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f652b7-aa59-47e5-8747-e74340d3f4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:53.521599 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f652b7-aa59-47e5-8747-e74340d3f4e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:53.532809 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f652b7-aa59-47e5-8747-e74340d3f4e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:53.534318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f652b7-aa59-47e5-8747-e74340d3f4e6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:53.549445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-e5f652b7-aa59-47e5-8747-e74340d3f4e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:53.549445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:53.549445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:53.549445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:53.549445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:53.549445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:53.549445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:53.549445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:53.549445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:53.549445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:53.549445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:53.549445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:53.549445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:53.549445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:53.549445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:53.550611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:53.551513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:53.551513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:53.551513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:53.551513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:53.551513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 324/635] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:53.551812 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f652b7-aa59-47e5-8747-e74340d3f4e6 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:53.615832 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60e1e6-072a-4524-82d0-60ff740ddae8 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:53.633816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:53.635346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:53.651337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996f919-26ca-4e78-9459-ed34e3413e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:53.651987 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996f919-26ca-4e78-9459-ed34e3413e51 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:53.652650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996f919-26ca-4e78-9459-ed34e3413e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:53.652650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996f919-26ca-4e78-9459-ed34e3413e51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:53.664058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996f919-26ca-4e78-9459-ed34e3413e51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:53.665525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996f919-26ca-4e78-9459-ed34e3413e51 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:53.666652 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f652b7-aa59-47e5-8747-e74340d3f4e6 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:53.680855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-c996f919-26ca-4e78-9459-ed34e3413e51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:53.680855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:53.680855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:53.680855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:53.680855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:53.680855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:53.680855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:53.680855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:53.680855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:53.680855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:53.680855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:53.680855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:53.680855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:53.680855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:53.680855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:53.682085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:53.683096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:53.683096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:53.683096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:53.683096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:53.683096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 312/636] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:53.683374 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996f919-26ca-4e78-9459-ed34e3413e51 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:53.711167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484f730-0512-48e4-8e64-5804d60ec0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:53.711833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484f730-0512-48e4-8e64-5804d60ec0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:53.712116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484f730-0512-48e4-8e64-5804d60ec0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:53.712563 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484f730-0512-48e4-8e64-5804d60ec0b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:53.724781 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7484f730-0512-48e4-8e64-5804d60ec0b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:53.734777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-7484f730-0512-48e4-8e64-5804d60ec0b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:53.734777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:53.734777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:53.734777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:53.734777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:53.734777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:53.734777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:53.734777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:53.734777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:53.734777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:53.734777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:53.734777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:53.734777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:53.734777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:53.734777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:53.734777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:53.735810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:53.735810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:53.735810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:53.735810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:53.735810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:53.735810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:53.735810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:53.735810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:53.735810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:53.735810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:53.735810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:53.735810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:53.735810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:53.735810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:53.735810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:53.735810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:53.736662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:53.736662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:53.736662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:53.736662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:53.736662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:53.736662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:53.736662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:53.736662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:53.736662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:53.736662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:53.736662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:53.736662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:53.736662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 325/637] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:53.738535 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484f730-0512-48e4-8e64-5804d60ec0b5 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:53.830920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996f919-26ca-4e78-9459-ed34e3413e51 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:53.880304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e524798-5a34-4ba7-9351-2a753a191ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:53.881092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e524798-5a34-4ba7-9351-2a753a191ea5 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:53.881444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e524798-5a34-4ba7-9351-2a753a191ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:53.881809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e524798-5a34-4ba7-9351-2a753a191ea5 None admin] PATH_INFO: `/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/25a26f54accf4264aef646c3757de7e9/roles/624a5f7ed1cf440cb177291e89bb11ff` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:53.882110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484f730-0512-48e4-8e64-5804d60ec0b5 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:53.899655 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e524798-5a34-4ba7-9351-2a753a191ea5 None admin] RBAC: Authorizing `identity:create_grant(project_id=218a9ee5cb2842248b6995d2562bb74b, user_id=25a26f54accf4264aef646c3757de7e9, role_id=624a5f7ed1cf440cb177291e89bb11ff)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:53.901255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e524798-5a34-4ba7-9351-2a753a191ea5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:53.916843 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 313/638] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:10:53 2022] PUT /identity/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/25a26f54accf4264aef646c3757de7e9/roles/624a5f7ed1cf440cb177291e89bb11ff => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:53.922114 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e524798-5a34-4ba7-9351-2a753a191ea5 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:53.927294 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676f1fe-3bde-4e70-9cd4-d34e15d1a80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:53.927294 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676f1fe-3bde-4e70-9cd4-d34e15d1a80f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:53.927294 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676f1fe-3bde-4e70-9cd4-d34e15d1a80f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:53.927294 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676f1fe-3bde-4e70-9cd4-d34e15d1a80f None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:53.927740 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676f1fe-3bde-4e70-9cd4-d34e15d1a80f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:53.927740 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676f1fe-3bde-4e70-9cd4-d34e15d1a80f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:53.927740 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676f1fe-3bde-4e70-9cd4-d34e15d1a80f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:53.929181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:53.945580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 326/639] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:53.945580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676f1fe-3bde-4e70-9cd4-d34e15d1a80f None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:53.953047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:54.048002 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e524798-5a34-4ba7-9351-2a753a191ea5 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:54.063754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676f1fe-3bde-4e70-9cd4-d34e15d1a80f None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:54.091372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d83ccf-7587-4cda-a341-42c4fe9a5a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:54.092048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d83ccf-7587-4cda-a341-42c4fe9a5a94 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:54.093015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d83ccf-7587-4cda-a341-42c4fe9a5a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:54.093015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d83ccf-7587-4cda-a341-42c4fe9a5a94 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:54.093414 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d83ccf-7587-4cda-a341-42c4fe9a5a94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:54.093722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d83ccf-7587-4cda-a341-42c4fe9a5a94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:54.095282 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d83ccf-7587-4cda-a341-42c4fe9a5a94 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:54.108972 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 314/640] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:54.110360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedbcd7-5d4f-46be-b214-fb13dd03f7ac None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:54.110647 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedbcd7-5d4f-46be-b214-fb13dd03f7ac None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:54.110930 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeedbcd7-5d4f-46be-b214-fb13dd03f7ac None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:54.112286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 315/641] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:54.114605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75b21c-070d-45e4-b385-9d218dd242e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:54.114742 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeedbcd7-5d4f-46be-b214-fb13dd03f7ac None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:54.115530 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75b21c-070d-45e4-b385-9d218dd242e4 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:54.116007 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75b21c-070d-45e4-b385-9d218dd242e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:54.116417 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be75b21c-070d-45e4-b385-9d218dd242e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:54.117333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75b21c-070d-45e4-b385-9d218dd242e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:54.117808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75b21c-070d-45e4-b385-9d218dd242e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:54.119329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be75b21c-070d-45e4-b385-9d218dd242e4 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:54.129161 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 327/642] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:10:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:54.130232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75b21c-070d-45e4-b385-9d218dd242e4 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:54.190567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:54.199359 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:54.222412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeedbcd7-5d4f-46be-b214-fb13dd03f7ac None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:54.236495 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be75b21c-070d-45e4-b385-9d218dd242e4 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:54.259282 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251568f4-9263-4028-905e-d58ca7a636cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:54.259930 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251568f4-9263-4028-905e-d58ca7a636cc None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:54.260220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251568f4-9263-4028-905e-d58ca7a636cc None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:54.260502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251568f4-9263-4028-905e-d58ca7a636cc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:54.268889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251568f4-9263-4028-905e-d58ca7a636cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:54.270515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251568f4-9263-4028-905e-d58ca7a636cc None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:54.272298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd282644-fb94-4fca-97db-132f91d9fa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:54.272925 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd282644-fb94-4fca-97db-132f91d9fa90 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:54.273221 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd282644-fb94-4fca-97db-132f91d9fa90 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:54.273483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd282644-fb94-4fca-97db-132f91d9fa90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:54.280794 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd282644-fb94-4fca-97db-132f91d9fa90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:54.282356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd282644-fb94-4fca-97db-132f91d9fa90 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:54.283366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-251568f4-9263-4028-905e-d58ca7a636cc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:10:54.283366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:54.283366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:54.283366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:54.283366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:54.283366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:54.283366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:54.283366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:54.283366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:54.283366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:54.283366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:54.283366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:54.283366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:10:54.283366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:10:54.283366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:10:54.283366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:54.284323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:54.285275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:54.285275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:54.285275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:10:54.285275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:10:54.285275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:10:54.285275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:10:54.285275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:10:54.285275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:10:54.285275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:10:54.285275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:54.292676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 316/643] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:54.292676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754bc84-115e-48fe-a6f0-d715e3ac74ab None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:54.292676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754bc84-115e-48fe-a6f0-d715e3ac74ab None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:54.292676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754bc84-115e-48fe-a6f0-d715e3ac74ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:54.297174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-bd282644-fb94-4fca-97db-132f91d9fa90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:54.297174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:54.297174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:54.297174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:54.297174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:54.297174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:54.297174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:54.297174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:54.297174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:54.297174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:54.297174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:54.297174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:54.297174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:10:54.297174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:10:54.297174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:54.298144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:54.299204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:10:54.299204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:10:54.299204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:10:54.299204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:10:54.299204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:10:54.299204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:10:54.299204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:54.299204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:54.303157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 328/644] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:54.303990 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd282644-fb94-4fca-97db-132f91d9fa90 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:54.308365 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-1754bc84-115e-48fe-a6f0-d715e3ac74ab None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:54.309557 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1754bc84-115e-48fe-a6f0-d715e3ac74ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:54.380689 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 317/645] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:54.381930 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754bc84-115e-48fe-a6f0-d715e3ac74ab None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:54.420259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd282644-fb94-4fca-97db-132f91d9fa90 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:54.438054 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:54.454757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:54.458982 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700dbfa-309e-4386-b363-673f71d98de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:54.459639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700dbfa-309e-4386-b363-673f71d98de8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:54.459918 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700dbfa-309e-4386-b363-673f71d98de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:54.460190 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700dbfa-309e-4386-b363-673f71d98de8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:54.471437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700dbfa-309e-4386-b363-673f71d98de8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:54.472905 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700dbfa-309e-4386-b363-673f71d98de8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:54.488046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-3700dbfa-309e-4386-b363-673f71d98de8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:54.488046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:54.488046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:54.488046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:54.488046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:54.488046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:54.488046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:54.488046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:54.488046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:54.488046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:54.488046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:54.488046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:54.488046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:54.488046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:54.488046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:54.489177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:54.490103 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:54.490103 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:54.490103 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:54.490103 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:54.490103 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 329/646] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:54.490103 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700dbfa-309e-4386-b363-673f71d98de8 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:54.499967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754bc84-115e-48fe-a6f0-d715e3ac74ab None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:54.536747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bc33e-abbf-4ff1-8dd9-e9d71ca03495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:54.537528 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bc33e-abbf-4ff1-8dd9-e9d71ca03495 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:54.537942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bc33e-abbf-4ff1-8dd9-e9d71ca03495 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:54.538314 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bc33e-abbf-4ff1-8dd9-e9d71ca03495 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:54.539245 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225bc33e-abbf-4ff1-8dd9-e9d71ca03495 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:54.539664 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225bc33e-abbf-4ff1-8dd9-e9d71ca03495 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:54.541226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225bc33e-abbf-4ff1-8dd9-e9d71ca03495 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:54.563448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 318/647] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 15:10:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:54.563448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bc33e-abbf-4ff1-8dd9-e9d71ca03495 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:54.605138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700dbfa-309e-4386-b363-673f71d98de8 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:54.643011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83260828-5801-4215-981b-5d51a4738113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:54.643657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83260828-5801-4215-981b-5d51a4738113 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:54.644081 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83260828-5801-4215-981b-5d51a4738113 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:54.644368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83260828-5801-4215-981b-5d51a4738113 None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:54.645136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83260828-5801-4215-981b-5d51a4738113 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:54.645499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83260828-5801-4215-981b-5d51a4738113 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:54.647278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83260828-5801-4215-981b-5d51a4738113 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:54.660564 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 330/648] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:10:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:54.661505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83260828-5801-4215-981b-5d51a4738113 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:54.674244 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bc33e-abbf-4ff1-8dd9-e9d71ca03495 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:54.692389 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:54.703422 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:54.712412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac26352-95f1-47ef-91a6-23586ab4c879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:54.713136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac26352-95f1-47ef-91a6-23586ab4c879 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:54.713475 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac26352-95f1-47ef-91a6-23586ab4c879 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:54.713856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac26352-95f1-47ef-91a6-23586ab4c879 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:54.724721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac26352-95f1-47ef-91a6-23586ab4c879 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:54.729931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-fac26352-95f1-47ef-91a6-23586ab4c879 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:54.729931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:54.729931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:54.729931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:54.729931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:54.729931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:54.729931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:54.729931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:54.729931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:54.729931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:54.729931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:54.729931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:54.729931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:54.729931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:54.729931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:54.729931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:54.731033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:54.731033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:54.731033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:54.731033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:54.731033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:54.731033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:54.731033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:54.731033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:54.731033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:54.731033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:54.731033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:54.731033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:54.731033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:54.731033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:54.731033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:54.731033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:54.732080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:54.732080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:54.732080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:54.732080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:54.732080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:54.732080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:54.732080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:54.732080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:54.732080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:54.732080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:54.732080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:54.732080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:54.733614 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 319/649] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:54.734417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac26352-95f1-47ef-91a6-23586ab4c879 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:54.773714 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83260828-5801-4215-981b-5d51a4738113 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:54.807250 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f612e-6459-457a-982e-4fc140368cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:54.808144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f612e-6459-457a-982e-4fc140368cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:54.808255 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f612e-6459-457a-982e-4fc140368cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:54.808451 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f612e-6459-457a-982e-4fc140368cf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:54.809148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f612e-6459-457a-982e-4fc140368cf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:54.809452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f612e-6459-457a-982e-4fc140368cf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:54.810880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f612e-6459-457a-982e-4fc140368cf6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:54.826922 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 331/650] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:54.826922 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f612e-6459-457a-982e-4fc140368cf6 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:54.842372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac26352-95f1-47ef-91a6-23586ab4c879 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:54.884146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123e004-abe2-4057-8e6e-ca378932900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:54.884791 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123e004-abe2-4057-8e6e-ca378932900b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:54.885062 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123e004-abe2-4057-8e6e-ca378932900b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:54.885338 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123e004-abe2-4057-8e6e-ca378932900b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:54.913443 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:54.926980 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f612e-6459-457a-982e-4fc140368cf6 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:54.945829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:54.959222 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123e004-abe2-4057-8e6e-ca378932900b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:54.960813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123e004-abe2-4057-8e6e-ca378932900b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:54.963384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0efd7-b338-463b-b651-84ef05d70798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:54.964268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0efd7-b338-463b-b651-84ef05d70798 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:54.964404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0efd7-b338-463b-b651-84ef05d70798 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:54.964610 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0efd7-b338-463b-b651-84ef05d70798 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:54.965452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0efd7-b338-463b-b651-84ef05d70798 None admin] RBAC: Adding query filter params (role.id=624a5f7ed1cf440cb177291e89bb11ff, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=2de0ff99a2b74979a28a8b10e9ebdfcd) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:54.965762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0efd7-b338-463b-b651-84ef05d70798 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:54.967222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d0efd7-b338-463b-b651-84ef05d70798 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:54.973600 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 332/651] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:10:54 2022] GET /identity/v3/role_assignments?user.id=2de0ff99a2b74979a28a8b10e9ebdfcd&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=624a5f7ed1cf440cb177291e89bb11ff => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:54.974360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0efd7-b338-463b-b651-84ef05d70798 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:55.105189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 320/652] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:55.106138 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123e004-abe2-4057-8e6e-ca378932900b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:55.124485 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0efd7-b338-463b-b651-84ef05d70798 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:55.170609 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f0a3b-276a-4f9f-9988-708795ab0379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:55.171815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f0a3b-276a-4f9f-9988-708795ab0379 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:55.172099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f0a3b-276a-4f9f-9988-708795ab0379 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:55.172414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f0a3b-276a-4f9f-9988-708795ab0379 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:55.173121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f0a3b-276a-4f9f-9988-708795ab0379 None admin] RBAC: Adding query filter params (name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:55.173430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f0a3b-276a-4f9f-9988-708795ab0379 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:55.175008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f0a3b-276a-4f9f-9988-708795ab0379 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:55.184033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 333/653] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:10:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:55.184964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f0a3b-276a-4f9f-9988-708795ab0379 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:55.200882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:55.215394 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:55.233035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123e004-abe2-4057-8e6e-ca378932900b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:55.282886 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256892e-db50-4a33-b87d-76e54c7b2e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:55.283545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256892e-db50-4a33-b87d-76e54c7b2e31 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:55.283821 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256892e-db50-4a33-b87d-76e54c7b2e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:55.284093 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256892e-db50-4a33-b87d-76e54c7b2e31 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:55.292015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256892e-db50-4a33-b87d-76e54c7b2e31 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:55.293554 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256892e-db50-4a33-b87d-76e54c7b2e31 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:55.306199 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f0a3b-276a-4f9f-9988-708795ab0379 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:55.306917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e256892e-db50-4a33-b87d-76e54c7b2e31 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:10:55.306917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:55.306917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:55.306917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:55.306917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:55.306917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:55.306917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:55.306917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:55.306917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:55.306917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:55.306917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:55.306917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:55.306917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:10:55.306917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:10:55.306917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:10:55.306917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:55.307852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:55.309305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:55.309305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:55.309305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:10:55.309305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:10:55.309305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:10:55.309305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:10:55.309305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:10:55.309305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:10:55.309305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:10:55.309305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:55.309305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 321/654] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:55.309305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256892e-db50-4a33-b87d-76e54c7b2e31 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:55.349873 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708e5de-62b8-4614-85ec-d6d605af4808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:55.350952 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708e5de-62b8-4614-85ec-d6d605af4808 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:55.351439 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708e5de-62b8-4614-85ec-d6d605af4808 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:55.351899 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708e5de-62b8-4614-85ec-d6d605af4808 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:55.364280 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708e5de-62b8-4614-85ec-d6d605af4808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:55.366093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708e5de-62b8-4614-85ec-d6d605af4808 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:55.382016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-e708e5de-62b8-4614-85ec-d6d605af4808 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:55.382016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:55.382016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:55.382016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:55.382016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:55.382016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:55.382016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:55.382016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:55.382016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:55.382016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:55.382016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:55.382016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:55.382016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:55.382016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:55.382016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:55.383162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:55.384175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:55.384175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:55.384175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:55.384175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:55.390734 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 334/655] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:55.391998 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708e5de-62b8-4614-85ec-d6d605af4808 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:55.440439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256892e-db50-4a33-b87d-76e54c7b2e31 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:55.476548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cab2d1-5ac4-42d7-a211-6ee80ac37c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:55.477554 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cab2d1-5ac4-42d7-a211-6ee80ac37c72 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:55.478051 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cab2d1-5ac4-42d7-a211-6ee80ac37c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:55.478513 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cab2d1-5ac4-42d7-a211-6ee80ac37c72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:55.480948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:55.491673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cab2d1-5ac4-42d7-a211-6ee80ac37c72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:55.493453 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cab2d1-5ac4-42d7-a211-6ee80ac37c72 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:55.503138 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:55.511260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-38cab2d1-5ac4-42d7-a211-6ee80ac37c72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:55.511260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:55.511260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:55.511260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:55.511260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:55.511260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:55.511260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:55.511260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:55.511260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:55.511260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:55.511260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:55.511260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:55.511260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:55.511260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:55.511260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:55.512162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:55.513034 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:55.513034 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:55.513034 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:55.513034 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:55.519926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 322/656] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:55.521253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cab2d1-5ac4-42d7-a211-6ee80ac37c72 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:55.525206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708e5de-62b8-4614-85ec-d6d605af4808 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:55.560743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7e4fa-c2bd-45c0-975b-ccde024ce5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:55.561803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7e4fa-c2bd-45c0-975b-ccde024ce5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:55.562277 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7e4fa-c2bd-45c0-975b-ccde024ce5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:55.562735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7e4fa-c2bd-45c0-975b-ccde024ce5a8 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:55.563768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7e4fa-c2bd-45c0-975b-ccde024ce5a8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:55.564279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7e4fa-c2bd-45c0-975b-ccde024ce5a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:55.573804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7e4fa-c2bd-45c0-975b-ccde024ce5a8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:55.585850 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 335/657] 10.222.0.48 () {56 vars in 1137 bytes} [Mon Aug 1 15:10:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:55.586732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7e4fa-c2bd-45c0-975b-ccde024ce5a8 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:55.644364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cab2d1-5ac4-42d7-a211-6ee80ac37c72 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:55.680810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d20350-deda-429b-b33e-c5160ff2715a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:55.681587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d20350-deda-429b-b33e-c5160ff2715a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:55.681770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d20350-deda-429b-b33e-c5160ff2715a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:55.682079 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d20350-deda-429b-b33e-c5160ff2715a None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:55.682990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d20350-deda-429b-b33e-c5160ff2715a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:55.683372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d20350-deda-429b-b33e-c5160ff2715a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:55.684772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d20350-deda-429b-b33e-c5160ff2715a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:55.689462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7e4fa-c2bd-45c0-975b-ccde024ce5a8 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:55.698161 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 323/658] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:55.699201 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d20350-deda-429b-b33e-c5160ff2715a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:55.714348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:55.725536 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfcb78-ec3e-4705-a343-f7dad17c7d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:55.726091 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfcb78-ec3e-4705-a343-f7dad17c7d7f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:55.726420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfcb78-ec3e-4705-a343-f7dad17c7d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:55.726687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfcb78-ec3e-4705-a343-f7dad17c7d7f None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:55.727448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfcb78-ec3e-4705-a343-f7dad17c7d7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:55.727734 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfcb78-ec3e-4705-a343-f7dad17c7d7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:55.729305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdfcb78-ec3e-4705-a343-f7dad17c7d7f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:55.742782 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 336/659] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:55.743719 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfcb78-ec3e-4705-a343-f7dad17c7d7f None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:55.768580 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:55.815927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d20350-deda-429b-b33e-c5160ff2715a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:55.847811 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfcb78-ec3e-4705-a343-f7dad17c7d7f None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:55.856709 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1854c-2120-4415-b8f1-79b70399b785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:55.858138 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1854c-2120-4415-b8f1-79b70399b785 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:55.858608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1854c-2120-4415-b8f1-79b70399b785 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:55.859042 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1854c-2120-4415-b8f1-79b70399b785 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:55.884933 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f959c-e29b-4120-a1f6-ec76cef65970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:55.885587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f959c-e29b-4120-a1f6-ec76cef65970 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:55.885958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f959c-e29b-4120-a1f6-ec76cef65970 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:55.886231 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f959c-e29b-4120-a1f6-ec76cef65970 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:55.893746 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f959c-e29b-4120-a1f6-ec76cef65970 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:55.895349 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f959c-e29b-4120-a1f6-ec76cef65970 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:55.907275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-480f959c-e29b-4120-a1f6-ec76cef65970 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:55.907275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:55.907275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:55.907275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:55.907275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:55.907275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:55.907275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:55.907275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:55.907275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:55.907275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:55.907275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:55.907275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:55.907275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:10:55.907275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:10:55.907275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:55.908379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:55.909396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:10:55.909396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:10:55.909396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:10:55.909396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:10:55.909396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:10:55.909396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:10:55.909396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:55.909396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:55.909396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 337/660] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:55.909903 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f959c-e29b-4120-a1f6-ec76cef65970 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:55.952525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1854c-2120-4415-b8f1-79b70399b785 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:55.954040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1854c-2120-4415-b8f1-79b70399b785 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:55.957626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:56.014409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f959c-e29b-4120-a1f6-ec76cef65970 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:56.015961 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:56.052322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a120611-622d-4507-8017-a5d5b88edeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:56.052974 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a120611-622d-4507-8017-a5d5b88edeec None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:56.053247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a120611-622d-4507-8017-a5d5b88edeec None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:56.053520 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a120611-622d-4507-8017-a5d5b88edeec None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:56.061285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 324/661] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:56.061445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a120611-622d-4507-8017-a5d5b88edeec None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:56.062774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a120611-622d-4507-8017-a5d5b88edeec None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:56.066558 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1854c-2120-4415-b8f1-79b70399b785 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:56.074983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-0a120611-622d-4507-8017-a5d5b88edeec None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:10:56.074983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:56.074983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:56.074983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:56.074983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:56.074983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:56.074983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:56.074983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:56.074983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:56.074983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:56.074983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:56.074983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:56.074983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:10:56.074983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:10:56.074983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:10:56.074983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:56.076119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:56.077061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:56.077061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:56.077061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:10:56.077061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:10:56.077061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:10:56.077061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:10:56.077061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:10:56.077061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:10:56.077061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:10:56.077061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:56.077061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 338/662] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:56.077796 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6fcf2-c6b4-46e1-a119-9429c069a9ea None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:56.077796 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6fcf2-c6b4-46e1-a119-9429c069a9ea None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:56.077796 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6fcf2-c6b4-46e1-a119-9429c069a9ea None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:56.081901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 339/663] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:56.081901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6fcf2-c6b4-46e1-a119-9429c069a9ea None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:56.182179 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1854c-2120-4415-b8f1-79b70399b785 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:56.194906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6fcf2-c6b4-46e1-a119-9429c069a9ea None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:56.198410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:56.219591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7e164-8d8d-41b8-b8b5-53c6c11c7eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:56.220406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7e164-8d8d-41b8-b8b5-53c6c11c7eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:56.220701 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7e164-8d8d-41b8-b8b5-53c6c11c7eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:56.220959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7e164-8d8d-41b8-b8b5-53c6c11c7eb7 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:56.222322 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7e164-8d8d-41b8-b8b5-53c6c11c7eb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:56.222615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7e164-8d8d-41b8-b8b5-53c6c11c7eb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:56.224214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7e164-8d8d-41b8-b8b5-53c6c11c7eb7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:56.232780 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66400b4-561d-4acf-b4ea-8ef7de0cb6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:56.233440 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66400b4-561d-4acf-b4ea-8ef7de0cb6cd None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:56.233718 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66400b4-561d-4acf-b4ea-8ef7de0cb6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:56.234026 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66400b4-561d-4acf-b4ea-8ef7de0cb6cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:56.237388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 325/664] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:10:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:56.238495 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7e164-8d8d-41b8-b8b5-53c6c11c7eb7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:56.248233 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66400b4-561d-4acf-b4ea-8ef7de0cb6cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:56.249752 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66400b4-561d-4acf-b4ea-8ef7de0cb6cd None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:56.265522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-a66400b4-561d-4acf-b4ea-8ef7de0cb6cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:56.265522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:56.265522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:56.265522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:56.265522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:56.265522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:56.265522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:56.265522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:56.265522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:56.265522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:56.265522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:56.265522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:56.265522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:56.265522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:56.265522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:56.266617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:56.267513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:56.267513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:56.267513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:56.267513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:56.267513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 340/665] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:56.268327 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80777cfb-9e22-43aa-91b8-ee54d3833da8 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:56.268615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80777cfb-9e22-43aa-91b8-ee54d3833da8 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:56.268868 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80777cfb-9e22-43aa-91b8-ee54d3833da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:56.272991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:56.289206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-80777cfb-9e22-43aa-91b8-ee54d3833da8 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:56.290463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-80777cfb-9e22-43aa-91b8-ee54d3833da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:56.360734 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7e164-8d8d-41b8-b8b5-53c6c11c7eb7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:56.365824 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 341/666] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:56.365824 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80777cfb-9e22-43aa-91b8-ee54d3833da8 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:56.405345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3e068-c27b-423e-ae6a-234732349e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:56.406350 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3e068-c27b-423e-ae6a-234732349e16 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:56.406842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3e068-c27b-423e-ae6a-234732349e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:56.407274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3e068-c27b-423e-ae6a-234732349e16 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:56.408221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3e068-c27b-423e-ae6a-234732349e16 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:56.408668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3e068-c27b-423e-ae6a-234732349e16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:56.410515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3e068-c27b-423e-ae6a-234732349e16 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:56.437631 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 326/667] 10.222.0.48 () {56 vars in 1139 bytes} [Mon Aug 1 15:10:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:56.437631 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3e068-c27b-423e-ae6a-234732349e16 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:56.462984 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:56.479332 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80777cfb-9e22-43aa-91b8-ee54d3833da8 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:56.515196 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:56.517163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca148e8-050b-40f0-933c-ea335d9fa605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:56.517867 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca148e8-050b-40f0-933c-ea335d9fa605 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:56.518123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca148e8-050b-40f0-933c-ea335d9fa605 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:56.518377 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca148e8-050b-40f0-933c-ea335d9fa605 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:56.519195 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca148e8-050b-40f0-933c-ea335d9fa605 None admin] RBAC: Adding query filter params (role.id=624a5f7ed1cf440cb177291e89bb11ff, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=94f696dfe4b34c4aa19104b5f7ee6e70) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:56.519497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca148e8-050b-40f0-933c-ea335d9fa605 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:56.520966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca148e8-050b-40f0-933c-ea335d9fa605 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:56.538152 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 342/668] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:10:56 2022] GET /identity/v3/role_assignments?user.id=94f696dfe4b34c4aa19104b5f7ee6e70&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=624a5f7ed1cf440cb177291e89bb11ff => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:56.538351 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3e068-c27b-423e-ae6a-234732349e16 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:56.538979 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca148e8-050b-40f0-933c-ea335d9fa605 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:56.582615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f248826-6ba6-4cb0-96b9-2364f1ddff7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:56.583207 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f248826-6ba6-4cb0-96b9-2364f1ddff7a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:56.583466 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f248826-6ba6-4cb0-96b9-2364f1ddff7a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:56.583735 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f248826-6ba6-4cb0-96b9-2364f1ddff7a None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:56.584468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f248826-6ba6-4cb0-96b9-2364f1ddff7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:56.584763 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f248826-6ba6-4cb0-96b9-2364f1ddff7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:56.586251 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f248826-6ba6-4cb0-96b9-2364f1ddff7a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:56.604450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 327/669] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:56.605508 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f248826-6ba6-4cb0-96b9-2364f1ddff7a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:56.671541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca148e8-050b-40f0-933c-ea335d9fa605 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:56.712197 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e75f0-1da9-4d8a-8a47-596772637551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:56.712856 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e75f0-1da9-4d8a-8a47-596772637551 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:56.713135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e75f0-1da9-4d8a-8a47-596772637551 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:56.713406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e75f0-1da9-4d8a-8a47-596772637551 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:56.724823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e75f0-1da9-4d8a-8a47-596772637551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:56.730910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-b10e75f0-1da9-4d8a-8a47-596772637551 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:56.730910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:56.730910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:56.730910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:56.730910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:56.730910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:56.730910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:56.730910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:56.730910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:56.730910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:56.730910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:56.730910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:56.730910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:56.730910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:56.730910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:56.730910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:56.731833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:56.731833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:56.731833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:56.731833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:56.731833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:56.731833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:56.731833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:56.731833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:56.731833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:56.731833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:56.731833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:56.731833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:56.731833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:56.731833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:56.731833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:56.731833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:56.733043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:56.733043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:56.733043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:56.733043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:56.733043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:56.733043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:56.733043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:56.733043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:56.733043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:56.733043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:56.733043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:10:56.733043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:56.736311 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 343/670] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:10:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:56.737401 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e75f0-1da9-4d8a-8a47-596772637551 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:56.738488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f248826-6ba6-4cb0-96b9-2364f1ddff7a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:56.751920 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:56.782159 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc15f08-18ef-4a61-93e5-7a240eff9516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:56.783153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc15f08-18ef-4a61-93e5-7a240eff9516 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:56.783436 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc15f08-18ef-4a61-93e5-7a240eff9516 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:56.783706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc15f08-18ef-4a61-93e5-7a240eff9516 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:56.792857 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:56.872825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e75f0-1da9-4d8a-8a47-596772637551 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:56.883405 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc15f08-18ef-4a61-93e5-7a240eff9516 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:56.885330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc15f08-18ef-4a61-93e5-7a240eff9516 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:56.912757 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045429c-ce99-4a13-9a3f-004a42225c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:56.913513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045429c-ce99-4a13-9a3f-004a42225c42 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:56.913882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045429c-ce99-4a13-9a3f-004a42225c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:56.914254 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045429c-ce99-4a13-9a3f-004a42225c42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:56.921718 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045429c-ce99-4a13-9a3f-004a42225c42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:56.923380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045429c-ce99-4a13-9a3f-004a42225c42 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:56.935308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-9045429c-ce99-4a13-9a3f-004a42225c42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:56.935308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:56.935308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:56.935308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:56.935308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:56.935308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:56.935308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:56.935308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:56.935308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:56.935308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:56.935308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:56.935308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:56.935308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:10:56.935308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:10:56.935308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:56.936268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:56.937181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:10:56.937181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:10:56.937181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:10:56.937181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:10:56.937181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:10:56.937181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:10:56.937181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:56.937181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:56.937181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 344/671] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:56.937661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045429c-ce99-4a13-9a3f-004a42225c42 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:57.004048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 328/672] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:57.005003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc15f08-18ef-4a61-93e5-7a240eff9516 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:57.034821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:57.050113 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:57.071157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045429c-ce99-4a13-9a3f-004a42225c42 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:57.107034 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443fa03-4b26-4c14-899d-615de5d0c806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:57.108040 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443fa03-4b26-4c14-899d-615de5d0c806 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:57.108315 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443fa03-4b26-4c14-899d-615de5d0c806 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:57.108582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443fa03-4b26-4c14-899d-615de5d0c806 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:57.109306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443fa03-4b26-4c14-899d-615de5d0c806 None admin] RBAC: Adding query filter params (name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:57.109603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443fa03-4b26-4c14-899d-615de5d0c806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:57.111046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443fa03-4b26-4c14-899d-615de5d0c806 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:57.131996 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 345/673] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:10:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:57.132880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443fa03-4b26-4c14-899d-615de5d0c806 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:57.134509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc15f08-18ef-4a61-93e5-7a240eff9516 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:57.173839 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfbe3e-1a57-49a3-b865-04c4e8e70020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:57.174472 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfbe3e-1a57-49a3-b865-04c4e8e70020 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:57.174749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfbe3e-1a57-49a3-b865-04c4e8e70020 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:57.175017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfbe3e-1a57-49a3-b865-04c4e8e70020 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:57.175783 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfbe3e-1a57-49a3-b865-04c4e8e70020 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:57.176084 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfbe3e-1a57-49a3-b865-04c4e8e70020 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:57.177531 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfbe3e-1a57-49a3-b865-04c4e8e70020 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:57.189430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 329/674] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:10:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:57.190295 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfbe3e-1a57-49a3-b865-04c4e8e70020 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:57.246704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443fa03-4b26-4c14-899d-615de5d0c806 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:57.278461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:57.284239 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e39a6-8896-479b-b69d-f233de578731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:57.284967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e39a6-8896-479b-b69d-f233de578731 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:57.285341 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e39a6-8896-479b-b69d-f233de578731 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:57.285612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e39a6-8896-479b-b69d-f233de578731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:57.286564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:57.290459 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfbe3e-1a57-49a3-b865-04c4e8e70020 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:57.299186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e39a6-8896-479b-b69d-f233de578731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:57.300767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e39a6-8896-479b-b69d-f233de578731 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:57.316318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-658e39a6-8896-479b-b69d-f233de578731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:57.316318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:57.316318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:57.316318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:57.316318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:57.316318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:57.316318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:57.316318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:57.316318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:57.316318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:57.316318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:57.316318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:57.316318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:57.316318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:57.316318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:57.317257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:57.318539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:57.318539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:57.318539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:57.318539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:57.318539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 346/675] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:57.319033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b614a8-7248-4045-9be3-4ea7f5716e3c None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:57.319103 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b614a8-7248-4045-9be3-4ea7f5716e3c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:57.319321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b614a8-7248-4045-9be3-4ea7f5716e3c None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:57.325549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 347/676] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:57.325549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b614a8-7248-4045-9be3-4ea7f5716e3c None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:57.335983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ded82-de44-4a50-8275-05b3f38882b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:57.336664 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ded82-de44-4a50-8275-05b3f38882b8 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:57.336940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ded82-de44-4a50-8275-05b3f38882b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:57.337184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ded82-de44-4a50-8275-05b3f38882b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:57.349823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ded82-de44-4a50-8275-05b3f38882b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:57.351327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ded82-de44-4a50-8275-05b3f38882b8 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:57.366969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-547ded82-de44-4a50-8275-05b3f38882b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:57.366969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:57.366969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:57.366969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:57.366969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:57.366969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:57.366969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:57.366969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:57.366969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:57.366969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:57.366969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:57.366969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:57.366969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:57.366969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:57.366969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:57.367940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:57.369595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:57.369595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:57.369595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:57.369595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:57.369595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 330/677] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:57.369595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ded82-de44-4a50-8275-05b3f38882b8 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:57.445256 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b614a8-7248-4045-9be3-4ea7f5716e3c None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:57.481644 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ded82-de44-4a50-8275-05b3f38882b8 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:57.488969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a542b3f-c8b0-4516-b2ab-d5f73ffc13ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:57.489596 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a542b3f-c8b0-4516-b2ab-d5f73ffc13ae None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:57.489901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a542b3f-c8b0-4516-b2ab-d5f73ffc13ae None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:57.490177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a542b3f-c8b0-4516-b2ab-d5f73ffc13ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:57.490981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a542b3f-c8b0-4516-b2ab-d5f73ffc13ae None admin] RBAC: Adding query filter params (role.id=624a5f7ed1cf440cb177291e89bb11ff, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=63824ba9369c4899a4b946588cbdae24) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:57.491293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a542b3f-c8b0-4516-b2ab-d5f73ffc13ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:57.492746 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a542b3f-c8b0-4516-b2ab-d5f73ffc13ae None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:57.509820 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 348/678] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:10:57 2022] GET /identity/v3/role_assignments?user.id=63824ba9369c4899a4b946588cbdae24&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=624a5f7ed1cf440cb177291e89bb11ff => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:57.509820 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2715e-e148-4c64-b2ab-9d11c8c3c882 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:57.509820 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2715e-e148-4c64-b2ab-9d11c8c3c882 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:57.509820 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2715e-e148-4c64-b2ab-9d11c8c3c882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:57.527610 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-adc2715e-e148-4c64-b2ab-9d11c8c3c882 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:57.529571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-adc2715e-e148-4c64-b2ab-9d11c8c3c882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:57.534922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e064b05-a20a-42f3-80de-fc02852ea26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:57.535672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e064b05-a20a-42f3-80de-fc02852ea26a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:57.536019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e064b05-a20a-42f3-80de-fc02852ea26a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:57.536360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e064b05-a20a-42f3-80de-fc02852ea26a None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:57.537147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e064b05-a20a-42f3-80de-fc02852ea26a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:57.537521 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e064b05-a20a-42f3-80de-fc02852ea26a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:57.539131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e064b05-a20a-42f3-80de-fc02852ea26a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:57.558420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 331/679] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:57.559638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e064b05-a20a-42f3-80de-fc02852ea26a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:57.573829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:57.582653 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:57.622293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 349/680] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:57.623567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2715e-e148-4c64-b2ab-9d11c8c3c882 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:57.675879 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e064b05-a20a-42f3-80de-fc02852ea26a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:57.717206 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc5d13-e580-42e3-802c-7bf84e519dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:57.717970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc5d13-e580-42e3-802c-7bf84e519dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:57.718322 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc5d13-e580-42e3-802c-7bf84e519dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:57.718707 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc5d13-e580-42e3-802c-7bf84e519dd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:57.719536 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc5d13-e580-42e3-802c-7bf84e519dd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:57.719973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc5d13-e580-42e3-802c-7bf84e519dd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:57.721473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc5d13-e580-42e3-802c-7bf84e519dd7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:57.737229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 332/681] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:57.738153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc5d13-e580-42e3-802c-7bf84e519dd7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:57.755893 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2715e-e148-4c64-b2ab-9d11c8c3c882 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:57.798704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c9f30-9c96-45e7-a044-11bbf1dc0c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:57.799723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c9f30-9c96-45e7-a044-11bbf1dc0c66 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:57.800239 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c9f30-9c96-45e7-a044-11bbf1dc0c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:57.800745 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c9f30-9c96-45e7-a044-11bbf1dc0c66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:57.809332 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c9f30-9c96-45e7-a044-11bbf1dc0c66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:57.811688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c9f30-9c96-45e7-a044-11bbf1dc0c66 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:57.820990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:57.825912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-567c9f30-9c96-45e7-a044-11bbf1dc0c66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:57.825912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:57.825912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:57.825912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:57.825912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:57.825912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:57.825912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:57.825912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:57.825912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:57.825912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:57.825912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:57.825912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:57.825912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:10:57.825912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:10:57.825912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:57.826885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:57.827838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:10:57.827838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:10:57.827838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:10:57.827838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:10:57.827838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:10:57.827838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:10:57.827838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:57.827838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:57.827838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 350/682] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:57.828327 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c9f30-9c96-45e7-a044-11bbf1dc0c66 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:57.855087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc5d13-e580-42e3-802c-7bf84e519dd7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:57.879383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:57.896500 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941b780-f9aa-44b8-9364-90c8cd6122d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:57.897476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941b780-f9aa-44b8-9364-90c8cd6122d5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:57.898102 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941b780-f9aa-44b8-9364-90c8cd6122d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:57.898398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941b780-f9aa-44b8-9364-90c8cd6122d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:57.913163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941b780-f9aa-44b8-9364-90c8cd6122d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:10:57.920657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e941b780-f9aa-44b8-9364-90c8cd6122d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:10:57.920657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:57.920657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:57.920657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:57.920657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:57.920657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:57.920657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:57.920657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:57.920657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:57.920657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:57.920657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:57.920657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:57.920657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:10:57.920657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:10:57.920657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:10:57.920657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:10:57.921691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:10:57.921691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:10:57.921691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:10:57.921691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:10:57.921691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:57.921691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:57.921691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:57.921691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:57.921691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:57.921691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:57.921691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:57.921691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:57.921691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:57.921691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:57.921691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:57.921691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:57.922626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:57.922626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:57.922626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:57.922626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:57.922626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:10:57.922626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:10:57.922626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:10:57.922626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:10:57.922626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:10:57.922626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:10:57.922626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:10:57.922626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:57.923272 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 333/683] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:10:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:57.929806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941b780-f9aa-44b8-9364-90c8cd6122d5 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:57.942017 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c9f30-9c96-45e7-a044-11bbf1dc0c66 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:57.983571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb842c3-adaf-4a4a-8072-021a859689c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:57.984233 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb842c3-adaf-4a4a-8072-021a859689c3 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:57.984584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb842c3-adaf-4a4a-8072-021a859689c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:57.984892 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb842c3-adaf-4a4a-8072-021a859689c3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:57.993511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb842c3-adaf-4a4a-8072-021a859689c3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:57.995226 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb842c3-adaf-4a4a-8072-021a859689c3 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:58.008173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-bcb842c3-adaf-4a4a-8072-021a859689c3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:10:58.008173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:58.008173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:58.008173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:58.008173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:58.008173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:58.008173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:58.008173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:58.008173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:58.008173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:58.008173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:58.008173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:58.008173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:10:58.008173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:10:58.008173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:10:58.008173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:58.009113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:58.010116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:58.010116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:58.010116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:10:58.010116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:10:58.010116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:10:58.010116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:10:58.010116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:10:58.010116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:10:58.010116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:10:58.010116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:58.010116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 351/684] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:10:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:58.010710 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb842c3-adaf-4a4a-8072-021a859689c3 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:58.051274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941b780-f9aa-44b8-9364-90c8cd6122d5 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:58.088977 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:58.097199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992ac39-3039-45a3-b924-0f203dfc3d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:58.097864 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992ac39-3039-45a3-b924-0f203dfc3d4c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:58.098148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992ac39-3039-45a3-b924-0f203dfc3d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:58.098425 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992ac39-3039-45a3-b924-0f203dfc3d4c None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:58.099600 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992ac39-3039-45a3-b924-0f203dfc3d4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:58.100169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992ac39-3039-45a3-b924-0f203dfc3d4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:58.101900 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992ac39-3039-45a3-b924-0f203dfc3d4c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:58.123962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 334/685] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:10:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:58.123962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992ac39-3039-45a3-b924-0f203dfc3d4c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:58.125322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:58.132344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb842c3-adaf-4a4a-8072-021a859689c3 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:58.176726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a2f73-090c-4bbf-9ab5-e128f8bd25bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:58.177410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a2f73-090c-4bbf-9ab5-e128f8bd25bd None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:58.177692 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a2f73-090c-4bbf-9ab5-e128f8bd25bd None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:58.178009 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a2f73-090c-4bbf-9ab5-e128f8bd25bd None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:58.178746 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a2f73-090c-4bbf-9ab5-e128f8bd25bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:58.179482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a2f73-090c-4bbf-9ab5-e128f8bd25bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:58.181170 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a2f73-090c-4bbf-9ab5-e128f8bd25bd None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:58.192512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 352/686] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:10:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:58.193345 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a2f73-090c-4bbf-9ab5-e128f8bd25bd None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:58.240036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992ac39-3039-45a3-b924-0f203dfc3d4c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:58.277299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e7a94-e303-415b-a1e3-87d08705b3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:58.278349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e7a94-e303-415b-a1e3-87d08705b3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:58.278850 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e7a94-e303-415b-a1e3-87d08705b3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:58.279322 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e7a94-e303-415b-a1e3-87d08705b3e4 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:58.280299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e7a94-e303-415b-a1e3-87d08705b3e4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:58.280778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e7a94-e303-415b-a1e3-87d08705b3e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:58.282465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e7a94-e303-415b-a1e3-87d08705b3e4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:58.301621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 335/687] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 15:10:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:58.302888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e7a94-e303-415b-a1e3-87d08705b3e4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:58.312694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a2f73-090c-4bbf-9ab5-e128f8bd25bd None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:58.354809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:58.358057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f0f2c-39e4-4c5d-b13a-371ded0f9294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:58.358740 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f0f2c-39e4-4c5d-b13a-371ded0f9294 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:58.359019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f0f2c-39e4-4c5d-b13a-371ded0f9294 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:58.359294 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f0f2c-39e4-4c5d-b13a-371ded0f9294 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:58.360477 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f0f2c-39e4-4c5d-b13a-371ded0f9294 None admin] RBAC: Adding query filter params (role.id=624a5f7ed1cf440cb177291e89bb11ff, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=5ef6577b10f7426a925d594e627f27cf) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:58.360802 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f0f2c-39e4-4c5d-b13a-371ded0f9294 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:58.362821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9f0f2c-39e4-4c5d-b13a-371ded0f9294 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:58.373836 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 353/688] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:10:58 2022] GET /identity/v3/role_assignments?user.id=5ef6577b10f7426a925d594e627f27cf&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=624a5f7ed1cf440cb177291e89bb11ff => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:58.373836 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f0f2c-39e4-4c5d-b13a-371ded0f9294 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:58.403243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:58.476006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e7a94-e303-415b-a1e3-87d08705b3e4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:58.494406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f0f2c-39e4-4c5d-b13a-371ded0f9294 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:58.534942 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21854b-6a57-4d2d-9ebb-7f91f14c9fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:58.535753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21854b-6a57-4d2d-9ebb-7f91f14c9fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:58.535999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21854b-6a57-4d2d-9ebb-7f91f14c9fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:58.536267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21854b-6a57-4d2d-9ebb-7f91f14c9fb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:58.562630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef69815-5aa8-47ca-9bcc-caf0ae3afa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:58.563274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef69815-5aa8-47ca-9bcc-caf0ae3afa71 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:58.563561 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef69815-5aa8-47ca-9bcc-caf0ae3afa71 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:58.563833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef69815-5aa8-47ca-9bcc-caf0ae3afa71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:58.587615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef69815-5aa8-47ca-9bcc-caf0ae3afa71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:58.589271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef69815-5aa8-47ca-9bcc-caf0ae3afa71 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:58.616020 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21854b-6a57-4d2d-9ebb-7f91f14c9fb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:58.617682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21854b-6a57-4d2d-9ebb-7f91f14c9fb3 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:58.631228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-bef69815-5aa8-47ca-9bcc-caf0ae3afa71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:58.631228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:58.631228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:58.631228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:58.631228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:58.631228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:58.631228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:58.631228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:58.631228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:58.631228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:58.631228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:58.631228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:58.631228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:58.631228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:58.631228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:58.632188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:58.633986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:58.633986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:58.633986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:58.633986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:58.633986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 336/689] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:58.638544 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef69815-5aa8-47ca-9bcc-caf0ae3afa71 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:58.645672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:58.743305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 354/690] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:58.748514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21854b-6a57-4d2d-9ebb-7f91f14c9fb3 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:58.755394 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:58.759314 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef69815-5aa8-47ca-9bcc-caf0ae3afa71 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:58.804746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e4ee60-fc7c-4d2c-a9cd-f48523cd111c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:58.805697 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4ee60-fc7c-4d2c-a9cd-f48523cd111c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:58.806005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4ee60-fc7c-4d2c-a9cd-f48523cd111c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:58.806381 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4ee60-fc7c-4d2c-a9cd-f48523cd111c None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:58.807093 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e4ee60-fc7c-4d2c-a9cd-f48523cd111c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:58.807398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e4ee60-fc7c-4d2c-a9cd-f48523cd111c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:58.808850 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e4ee60-fc7c-4d2c-a9cd-f48523cd111c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:58.823210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 337/691] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:10:58.824429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d16e0-16a5-4a66-843d-4d4bee5c7422 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:58.824720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d16e0-16a5-4a66-843d-4d4bee5c7422 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:58.824984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d16e0-16a5-4a66-843d-4d4bee5c7422 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:58.832817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 338/692] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:58.832817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d16e0-16a5-4a66-843d-4d4bee5c7422 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:58.883586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21854b-6a57-4d2d-9ebb-7f91f14c9fb3 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:58.910627 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:58.923081 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc4d41-e686-46c9-a7fe-32abb0681047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:58.923860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc4d41-e686-46c9-a7fe-32abb0681047 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:58.924230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc4d41-e686-46c9-a7fe-32abb0681047 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:58.924762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc4d41-e686-46c9-a7fe-32abb0681047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:58.936064 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc4d41-e686-46c9-a7fe-32abb0681047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:58.937795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc4d41-e686-46c9-a7fe-32abb0681047 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:58.954656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-8abc4d41-e686-46c9-a7fe-32abb0681047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:58.954656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:58.954656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:58.954656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:58.954656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:58.954656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:58.954656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:58.954656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:58.954656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:58.954656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:58.954656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:58.954656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:58.954656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:58.954656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:58.954656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:58.955597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:58.956589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:58.956589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:58.956589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:58.956589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:10:58.960892 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 355/693] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:10:58.962416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda363df-7ab7-43fd-a1e7-a89289838318 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:58.962678 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda363df-7ab7-43fd-a1e7-a89289838318 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:58.963097 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda363df-7ab7-43fd-a1e7-a89289838318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:58.969402 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d16e0-16a5-4a66-843d-4d4bee5c7422 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:58.983636 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-cda363df-7ab7-43fd-a1e7-a89289838318 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:58.985017 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-cda363df-7ab7-43fd-a1e7-a89289838318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:59.011529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb372f6-57f4-4da5-9438-64a69f7a54f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:59.012289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb372f6-57f4-4da5-9438-64a69f7a54f4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:59.012895 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb372f6-57f4-4da5-9438-64a69f7a54f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:59.013028 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb372f6-57f4-4da5-9438-64a69f7a54f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:59.020873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb372f6-57f4-4da5-9438-64a69f7a54f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:59.022632 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb372f6-57f4-4da5-9438-64a69f7a54f4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:59.036474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-7eb372f6-57f4-4da5-9438-64a69f7a54f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:10:59.036474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:59.036474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:59.036474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:59.036474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:59.036474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:59.036474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:59.036474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:59.036474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:59.036474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:59.036474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:59.036474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:59.036474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:10:59.036474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:10:59.036474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:10:59.036474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:59.037497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:59.038656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:59.038656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:59.038656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:10:59.038656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:10:59.038656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:10:59.038656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:10:59.038656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:10:59.038656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:10:59.038656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:10:59.038656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:59.038656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 339/694] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 15:10:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:59.039255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb372f6-57f4-4da5-9438-64a69f7a54f4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:59.063587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:59.067946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 356/695] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:10:59.069039 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda363df-7ab7-43fd-a1e7-a89289838318 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:59.176348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda363df-7ab7-43fd-a1e7-a89289838318 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:59.176626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb372f6-57f4-4da5-9438-64a69f7a54f4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:59.184820 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:59.213655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fc0a7-f3e9-43b9-8319-57b384a8342d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:59.214331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fc0a7-f3e9-43b9-8319-57b384a8342d None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:59.214616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fc0a7-f3e9-43b9-8319-57b384a8342d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:59.214887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842fc0a7-f3e9-43b9-8319-57b384a8342d None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:59.215797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842fc0a7-f3e9-43b9-8319-57b384a8342d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:59.216113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842fc0a7-f3e9-43b9-8319-57b384a8342d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:59.216751 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432191d8-7b9d-43b3-b287-51b0f9af154d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:59.217586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842fc0a7-f3e9-43b9-8319-57b384a8342d None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:59.217857 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432191d8-7b9d-43b3-b287-51b0f9af154d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:59.218306 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432191d8-7b9d-43b3-b287-51b0f9af154d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:59.218747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432191d8-7b9d-43b3-b287-51b0f9af154d None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:59.219758 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432191d8-7b9d-43b3-b287-51b0f9af154d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:59.220328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432191d8-7b9d-43b3-b287-51b0f9af154d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:59.221932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432191d8-7b9d-43b3-b287-51b0f9af154d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:59.236241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 340/696] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:59.237454 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432191d8-7b9d-43b3-b287-51b0f9af154d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:59.241152 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 357/697] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 15:10:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:59.242784 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fc0a7-f3e9-43b9-8319-57b384a8342d None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:59.328105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:59.345717 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842fc0a7-f3e9-43b9-8319-57b384a8342d None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:59.351245 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432191d8-7b9d-43b3-b287-51b0f9af154d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:59.381463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc1700-f619-42ae-b7f2-25903f9a77ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:59.382137 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc1700-f619-42ae-b7f2-25903f9a77ec None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:59.382419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc1700-f619-42ae-b7f2-25903f9a77ec None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:59.382692 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc1700-f619-42ae-b7f2-25903f9a77ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:59.390743 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c18917-b15d-465f-aaab-92258514697b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:59.391865 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c18917-b15d-465f-aaab-92258514697b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:59.392343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c18917-b15d-465f-aaab-92258514697b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:59.392832 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c18917-b15d-465f-aaab-92258514697b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:59.404912 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c18917-b15d-465f-aaab-92258514697b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:59.406855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c18917-b15d-465f-aaab-92258514697b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:59.415980 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:59.424922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-92c18917-b15d-465f-aaab-92258514697b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:59.424922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:59.424922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:59.424922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:59.424922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:59.424922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:59.424922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:59.424922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:59.424922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:59.424922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:59.424922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:59.424922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:59.424922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:59.424922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:59.424922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:59.426130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:59.427108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:59.427108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:59.427108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:59.427108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:59.430144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 341/698] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:59.431214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c18917-b15d-465f-aaab-92258514697b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:59.456240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc1700-f619-42ae-b7f2-25903f9a77ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:59.457547 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc1700-f619-42ae-b7f2-25903f9a77ec None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:59.541042 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c18917-b15d-465f-aaab-92258514697b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:59.563555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 358/699] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:10:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:10:59.563555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc1700-f619-42ae-b7f2-25903f9a77ec None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:59.580049 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:59.587634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402a3bd-3457-4e5c-a484-764395b1a024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:59.588677 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402a3bd-3457-4e5c-a484-764395b1a024 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:59.589164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402a3bd-3457-4e5c-a484-764395b1a024 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:59.589638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402a3bd-3457-4e5c-a484-764395b1a024 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:59.597852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402a3bd-3457-4e5c-a484-764395b1a024 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:59.599668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c402a3bd-3457-4e5c-a484-764395b1a024 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:59.612919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-c402a3bd-3457-4e5c-a484-764395b1a024 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:59.612919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:59.612919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:59.612919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:59.612919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:59.612919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:59.612919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:59.612919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:59.612919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:59.612919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:59.612919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:59.612919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:59.612919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:10:59.612919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:10:59.612919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:59.613916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:59.615037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:10:59.615037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:10:59.615037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:10:59.615037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:10:59.615037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:10:59.615037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:10:59.615037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:10:59.615037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:59.617262 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 342/700] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:10:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:59.618298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402a3bd-3457-4e5c-a484-764395b1a024 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:59.646237 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:59.671030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc1700-f619-42ae-b7f2-25903f9a77ec None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:59.706341 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca05ce-e0b6-46da-a8ca-f0fe7604de7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:59.706957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca05ce-e0b6-46da-a8ca-f0fe7604de7d None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:59.707267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca05ce-e0b6-46da-a8ca-f0fe7604de7d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:59.707815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca05ce-e0b6-46da-a8ca-f0fe7604de7d None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:59.708219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca05ce-e0b6-46da-a8ca-f0fe7604de7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:59.708524 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca05ce-e0b6-46da-a8ca-f0fe7604de7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:59.709941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca05ce-e0b6-46da-a8ca-f0fe7604de7d None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:59.722410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 359/701] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:59.723219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca05ce-e0b6-46da-a8ca-f0fe7604de7d None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:59.730106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c402a3bd-3457-4e5c-a484-764395b1a024 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:59.769242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89e0b4-4ee6-4f98-a2da-869a6098f97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:59.769953 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89e0b4-4ee6-4f98-a2da-869a6098f97c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:59.770234 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89e0b4-4ee6-4f98-a2da-869a6098f97c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:59.770501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89e0b4-4ee6-4f98-a2da-869a6098f97c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:59.782349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd89e0b4-4ee6-4f98-a2da-869a6098f97c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:59.783829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd89e0b4-4ee6-4f98-a2da-869a6098f97c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:59.799167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-bd89e0b4-4ee6-4f98-a2da-869a6098f97c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:59.799167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:10:59.799167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:10:59.799167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:10:59.799167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:10:59.799167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:10:59.799167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:10:59.799167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:10:59.799167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:10:59.799167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:10:59.799167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:10:59.799167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:10:59.799167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:10:59.799167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:10:59.799167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:10:59.800280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:10:59.801215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:10:59.801215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:10:59.801215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:10:59.801215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:10:59.801215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 343/702] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:10:59.802201 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80d33b-fcac-43d7-8088-b69fea83613d None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:59.802484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80d33b-fcac-43d7-8088-b69fea83613d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:59.802758 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80d33b-fcac-43d7-8088-b69fea83613d None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:59.808433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 344/703] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:10:59.808433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80d33b-fcac-43d7-8088-b69fea83613d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:59.835598 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca05ce-e0b6-46da-a8ca-f0fe7604de7d None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:59.855557 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:59.869583 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:59.886950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff59a9-8cda-4611-b2ec-f5186db17cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:59.888063 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff59a9-8cda-4611-b2ec-f5186db17cdc None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:59.888411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff59a9-8cda-4611-b2ec-f5186db17cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:59.888666 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff59a9-8cda-4611-b2ec-f5186db17cdc None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:59.889442 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff59a9-8cda-4611-b2ec-f5186db17cdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:59.890174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff59a9-8cda-4611-b2ec-f5186db17cdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:59.891982 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff59a9-8cda-4611-b2ec-f5186db17cdc None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:59.905828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 360/704] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:10:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:59.905828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff59a9-8cda-4611-b2ec-f5186db17cdc None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:59.919477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80d33b-fcac-43d7-8088-b69fea83613d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:59.954640 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a78dd-a7fd-4526-b72c-c1ecd9d14f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:59.955401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a78dd-a7fd-4526-b72c-c1ecd9d14f14 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:59.955748 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a78dd-a7fd-4526-b72c-c1ecd9d14f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:59.956106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a78dd-a7fd-4526-b72c-c1ecd9d14f14 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:59.956827 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a78dd-a7fd-4526-b72c-c1ecd9d14f14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:59.958288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a78dd-a7fd-4526-b72c-c1ecd9d14f14 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:59.981296 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 345/705] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:10:59 2022] POST /identity/v3/users => generated 254 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:10:59.983843 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d902ea-6e6d-4a79-adc9-e155449ec9d4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:59.984135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d902ea-6e6d-4a79-adc9-e155449ec9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:59.984410 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d902ea-6e6d-4a79-adc9-e155449ec9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:00.007643 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-88d902ea-6e6d-4a79-adc9-e155449ec9d4 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:00.008908 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-88d902ea-6e6d-4a79-adc9-e155449ec9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:00.030032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff59a9-8cda-4611-b2ec-f5186db17cdc None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:00.080827 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff0524-21a2-40f9-a0c6-1767b7e942dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:00.082008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff0524-21a2-40f9-a0c6-1767b7e942dd None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:00.083157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff0524-21a2-40f9-a0c6-1767b7e942dd None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:00.083677 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff0524-21a2-40f9-a0c6-1767b7e942dd None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:00.084959 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff0524-21a2-40f9-a0c6-1767b7e942dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:00.085492 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff0524-21a2-40f9-a0c6-1767b7e942dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:00.088190 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff0524-21a2-40f9-a0c6-1767b7e942dd None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:00.099560 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 346/706] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:11:00.100776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d902ea-6e6d-4a79-adc9-e155449ec9d4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:00.110932 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 361/707] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:00.112362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff0524-21a2-40f9-a0c6-1767b7e942dd None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:00.148867 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:00.177675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:00.241380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d902ea-6e6d-4a79-adc9-e155449ec9d4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:00.241577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff0524-21a2-40f9-a0c6-1767b7e942dd None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:00.284917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829fdd2-d196-420b-9ca1-4956ced0852d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:00.285578 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829fdd2-d196-420b-9ca1-4956ced0852d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:00.285890 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829fdd2-d196-420b-9ca1-4956ced0852d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:00.286163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829fdd2-d196-420b-9ca1-4956ced0852d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:00.286967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829fdd2-d196-420b-9ca1-4956ced0852d None admin] RBAC: Adding query filter params (role.id=624a5f7ed1cf440cb177291e89bb11ff, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=25a26f54accf4264aef646c3757de7e9) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:00.287266 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829fdd2-d196-420b-9ca1-4956ced0852d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:00.288722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829fdd2-d196-420b-9ca1-4956ced0852d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:00.290676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd559aa-a959-4bf9-bdd4-135355f50786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:00.291804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd559aa-a959-4bf9-bdd4-135355f50786 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:00.292156 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd559aa-a959-4bf9-bdd4-135355f50786 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:00.292497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd559aa-a959-4bf9-bdd4-135355f50786 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:00.295857 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 347/708] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:11:00 2022] GET /identity/v3/role_assignments?user.id=25a26f54accf4264aef646c3757de7e9&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=624a5f7ed1cf440cb177291e89bb11ff => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:00.296930 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829fdd2-d196-420b-9ca1-4956ced0852d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:00.304833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd559aa-a959-4bf9-bdd4-135355f50786 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:00.307616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd559aa-a959-4bf9-bdd4-135355f50786 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:00.331704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-afd559aa-a959-4bf9-bdd4-135355f50786 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:00.331704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:00.331704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:00.331704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:00.331704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:00.331704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:00.331704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:00.331704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:00.331704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:00.331704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:00.331704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:00.331704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:00.331704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:11:00.331704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:11:00.331704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:00.332683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:00.333581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:11:00.333581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:11:00.333581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:11:00.333581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:11:00.333581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:11:00.333581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:11:00.333581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:00.333581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:00.347607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 362/709] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:00.354614 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd559aa-a959-4bf9-bdd4-135355f50786 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:00.480845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:00.490177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829fdd2-d196-420b-9ca1-4956ced0852d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:00.535449 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd559aa-a959-4bf9-bdd4-135355f50786 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:00.537674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24da07b-8091-4f34-992a-473b31bfc692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:00.539408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:00.539520 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24da07b-8091-4f34-992a-473b31bfc692 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:00.539726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24da07b-8091-4f34-992a-473b31bfc692 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:00.540451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24da07b-8091-4f34-992a-473b31bfc692 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:00.541099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24da07b-8091-4f34-992a-473b31bfc692 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:00.542705 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24da07b-8091-4f34-992a-473b31bfc692 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:00.557371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e24da07b-8091-4f34-992a-473b31bfc692 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:11:00.557371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:00.557371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:00.557371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:00.557371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:00.557371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:00.557371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:00.557371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:00.557371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:11:00.557371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:00.557371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:00.557371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:00.557371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:00.557371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:00.557371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:00.557371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:00.558476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:00.558476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:00.558476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:00.558476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:00.558476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:00.558476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:00.558476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:00.558476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:00.558476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:00.558476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:00.558476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:00.558476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:00.558476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:00.558476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:00.558476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:00.558476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:00.559364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:00.559364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:00.559364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:00.559364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:00.559364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:00.559364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:00.559364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:00.559364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:00.559364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:00.559364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:00.559364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:00.559364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:00.559364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:00.559364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:11:00.559364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:00.560184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 348/710] 10.222.0.48 () {56 vars in 1129 bytes} [Mon Aug 1 15:11:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:00.565562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24da07b-8091-4f34-992a-473b31bfc692 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:00.584520 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd593a7-db02-4a21-b630-8bc20a7788cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:00.586473 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd593a7-db02-4a21-b630-8bc20a7788cb None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:00.586962 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd593a7-db02-4a21-b630-8bc20a7788cb None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:00.587672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd593a7-db02-4a21-b630-8bc20a7788cb None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:00.588995 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd593a7-db02-4a21-b630-8bc20a7788cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:00.589912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd593a7-db02-4a21-b630-8bc20a7788cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:00.591593 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd593a7-db02-4a21-b630-8bc20a7788cb None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:00.610461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 363/711] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:11:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:00.618044 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd593a7-db02-4a21-b630-8bc20a7788cb None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:00.693464 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24da07b-8091-4f34-992a-473b31bfc692 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:00.731954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd593a7-db02-4a21-b630-8bc20a7788cb None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:00.740566 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3fed2-5e77-40ba-9e9b-c98e8f86c2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:00.741565 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3fed2-5e77-40ba-9e9b-c98e8f86c2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:00.742079 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3fed2-5e77-40ba-9e9b-c98e8f86c2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:00.742902 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3fed2-5e77-40ba-9e9b-c98e8f86c2e0 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:00.743797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3fed2-5e77-40ba-9e9b-c98e8f86c2e0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:00.744632 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3fed2-5e77-40ba-9e9b-c98e8f86c2e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:00.747597 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3fed2-5e77-40ba-9e9b-c98e8f86c2e0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:00.755053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 349/712] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 15:11:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:00.756784 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32421fd-b7d4-4f16-b0ca-d911e68045f3 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:00.757062 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32421fd-b7d4-4f16-b0ca-d911e68045f3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:00.757492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32421fd-b7d4-4f16-b0ca-d911e68045f3 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:00.763239 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 350/713] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:00.764339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32421fd-b7d4-4f16-b0ca-d911e68045f3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:00.779060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9a8f9-cd0e-451d-8f8e-1998a032b012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:00.779713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9a8f9-cd0e-451d-8f8e-1998a032b012 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:00.779959 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9a8f9-cd0e-451d-8f8e-1998a032b012 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:00.780238 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9a8f9-cd0e-451d-8f8e-1998a032b012 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:00.781079 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9a8f9-cd0e-451d-8f8e-1998a032b012 None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=2de0ff99a2b74979a28a8b10e9ebdfcd) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:00.781396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9a8f9-cd0e-451d-8f8e-1998a032b012 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:00.783093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9a8f9-cd0e-451d-8f8e-1998a032b012 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:00.789419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 364/714] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:11:00 2022] GET /identity/v3/role_assignments?user.id=2de0ff99a2b74979a28a8b10e9ebdfcd&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:00.790690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563745a-04d4-458b-8e61-235bdac74542 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:00.791032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563745a-04d4-458b-8e61-235bdac74542 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:00.791660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563745a-04d4-458b-8e61-235bdac74542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:00.794837 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:00.811971 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2563745a-04d4-458b-8e61-235bdac74542 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:00.813269 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2563745a-04d4-458b-8e61-235bdac74542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:00.839039 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:00.895840 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32421fd-b7d4-4f16-b0ca-d911e68045f3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:00.913378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 365/715] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:11:00.936843 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb779a-7a3a-4941-821a-332a0bcde5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:00.937885 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb779a-7a3a-4941-821a-332a0bcde5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:00.938314 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb779a-7a3a-4941-821a-332a0bcde5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:00.938728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb779a-7a3a-4941-821a-332a0bcde5d4 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:00.939625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb779a-7a3a-4941-821a-332a0bcde5d4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:00.940089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb779a-7a3a-4941-821a-332a0bcde5d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:00.941658 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb779a-7a3a-4941-821a-332a0bcde5d4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:00.948715 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 351/716] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 15:11:00 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:00.950057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563745a-04d4-458b-8e61-235bdac74542 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:01.071754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563745a-04d4-458b-8e61-235bdac74542 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:01.109483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cd985-bd2e-4904-8ade-ee8e9b81a9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:01.110527 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cd985-bd2e-4904-8ade-ee8e9b81a9ac None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:01.110953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cd985-bd2e-4904-8ade-ee8e9b81a9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:01.111370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cd985-bd2e-4904-8ade-ee8e9b81a9ac None admin] PATH_INFO: `/v3/services/network` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:01.112154 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183cd985-bd2e-4904-8ade-ee8e9b81a9ac None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:01.113722 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183cd985-bd2e-4904-8ade-ee8e9b81a9ac None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-183cd985-bd2e-4904-8ade-ee8e9b81a9ac None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:01.127737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:01.129019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:01.129019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:01.129019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:01.129019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:01.129019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:01.129019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:01.129019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:01.129019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:01.129019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:01.129019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:01.129019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:01.129019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:01.129019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:01.129019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:01.129019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:01.129019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:01.130083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:01.130083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:01.130083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:01.130083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:01.130083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:01.130083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:01.130083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:01.130083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:01.130083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:01.130083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:01.130083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:01.130083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:01.130083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:11:01.130083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:01.132894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 366/717] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:00 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:01.138803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb779a-7a3a-4941-821a-332a0bcde5d4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:01.254591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:01.273542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb779a-7a3a-4941-821a-332a0bcde5d4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:01.314129 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95551ca6-c3aa-47dd-87a6-372cf61f7762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:01.314824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95551ca6-c3aa-47dd-87a6-372cf61f7762 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:01.315106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95551ca6-c3aa-47dd-87a6-372cf61f7762 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:01.315381 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95551ca6-c3aa-47dd-87a6-372cf61f7762 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:01.316073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95551ca6-c3aa-47dd-87a6-372cf61f7762 None admin] RBAC: Adding query filter params (name=network) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:01.316374 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95551ca6-c3aa-47dd-87a6-372cf61f7762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:01.317765 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95551ca6-c3aa-47dd-87a6-372cf61f7762 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:01.322737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 352/718] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 15:11:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:01.328010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cd985-bd2e-4904-8ade-ee8e9b81a9ac None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:01.376008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:01.481886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cd985-bd2e-4904-8ade-ee8e9b81a9ac None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:01.525038 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fcf7c-027e-42d6-b8b5-ea0b7472166b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:01.525682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fcf7c-027e-42d6-b8b5-ea0b7472166b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:01.525979 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fcf7c-027e-42d6-b8b5-ea0b7472166b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:01.526277 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fcf7c-027e-42d6-b8b5-ea0b7472166b None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:01.527024 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fcf7c-027e-42d6-b8b5-ea0b7472166b None admin] RBAC: Adding query filter params (type=network) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:01.527329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fcf7c-027e-42d6-b8b5-ea0b7472166b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:01.528723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fcf7c-027e-42d6-b8b5-ea0b7472166b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:01.539420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 367/719] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 15:11:01 2022] GET /identity/v3/services?type=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:02.559494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94091dc-160e-415c-8948-e47180853e7a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:02.560058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94091dc-160e-415c-8948-e47180853e7a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:02.560058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94091dc-160e-415c-8948-e47180853e7a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:02.561637 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 353/720] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:02.567222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df29a1a-c864-407a-996b-7869b13900c5 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:02.567649 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df29a1a-c864-407a-996b-7869b13900c5 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:02.567820 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df29a1a-c864-407a-996b-7869b13900c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:02.586978 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-9df29a1a-c864-407a-996b-7869b13900c5 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:02.588208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9df29a1a-c864-407a-996b-7869b13900c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:02.678509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 368/721] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:11:02.712144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94091dc-160e-415c-8948-e47180853e7a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:02.848584 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94091dc-160e-415c-8948-e47180853e7a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:02.885236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b86948-84e2-4652-bb6c-a01b674d8d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:02.885925 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b86948-84e2-4652-bb6c-a01b674d8d9d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:02.886209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b86948-84e2-4652-bb6c-a01b674d8d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:02.886485 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b86948-84e2-4652-bb6c-a01b674d8d9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:02.897436 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b86948-84e2-4652-bb6c-a01b674d8d9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:02.902988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-88b86948-84e2-4652-bb6c-a01b674d8d9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:11:02.902988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:02.902988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:02.902988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:02.902988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:02.902988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:02.902988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:02.902988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:02.902988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:02.902988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:02.902988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:02.902988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:02.902988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:02.902988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:02.902988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:02.902988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:02.903945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:02.903945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:02.903945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:02.903945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:02.903945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:02.903945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:02.903945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:02.903945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:02.903945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:02.903945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:02.903945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:02.903945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:02.903945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:02.903945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:02.903945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:02.903945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:02.904794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:02.904794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:02.904794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:02.904794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:02.904794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:02.904794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:02.904794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:02.904794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:02.904794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:02.904794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:02.904794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:11:02.904794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:02.904794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 354/722] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:11:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:02.911155 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df29a1a-c864-407a-996b-7869b13900c5 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:02.945891 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:03.061926 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df29a1a-c864-407a-996b-7869b13900c5 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:03.081654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8a89b-cb13-4551-add8-5b5d90aafb4a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:03.082061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8a89b-cb13-4551-add8-5b5d90aafb4a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:03.082411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8a89b-cb13-4551-add8-5b5d90aafb4a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:03.084230 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 355/723] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:03.098115 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9e4d2-8b9c-433d-9b7c-a889a7bf441d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:03.098480 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9e4d2-8b9c-433d-9b7c-a889a7bf441d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:03.098908 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9e4d2-8b9c-433d-9b7c-a889a7bf441d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:03.111618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea84be-3d4d-4620-bd1c-a2673657c5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:03.112505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea84be-3d4d-4620-bd1c-a2673657c5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:03.112975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea84be-3d4d-4620-bd1c-a2673657c5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:03.113420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea84be-3d4d-4620-bd1c-a2673657c5c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:03.114353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea84be-3d4d-4620-bd1c-a2673657c5c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:03.114831 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea84be-3d4d-4620-bd1c-a2673657c5c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:03.116377 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea84be-3d4d-4620-bd1c-a2673657c5c9 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:03.129266 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 369/724] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:11:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:03.130364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-0bd9e4d2-8b9c-433d-9b7c-a889a7bf441d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:03.131578 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0bd9e4d2-8b9c-433d-9b7c-a889a7bf441d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:03.134715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea84be-3d4d-4620-bd1c-a2673657c5c9 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:03.138291 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:03.219014 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 356/725] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:11:03.220695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688b192-7eef-4f62-9b7d-5e387f71d795 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:03.221023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688b192-7eef-4f62-9b7d-5e387f71d795 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:03.221304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688b192-7eef-4f62-9b7d-5e387f71d795 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:03.222844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 357/726] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:03.228399 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729d246-7a47-493d-aab5-cc7f1ba8b550 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:03.228679 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729d246-7a47-493d-aab5-cc7f1ba8b550 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:03.228947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729d246-7a47-493d-aab5-cc7f1ba8b550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:03.235322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:03.254353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3729d246-7a47-493d-aab5-cc7f1ba8b550 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:03.255640 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3729d246-7a47-493d-aab5-cc7f1ba8b550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:03.264317 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea84be-3d4d-4620-bd1c-a2673657c5c9 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:03.301718 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fb76e-ccc0-45c6-89ed-63afa3cfdc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:03.302758 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fb76e-ccc0-45c6-89ed-63afa3cfdc3a None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:03.303260 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fb76e-ccc0-45c6-89ed-63afa3cfdc3a None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:03.303723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fb76e-ccc0-45c6-89ed-63afa3cfdc3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:03.317297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8fb76e-ccc0-45c6-89ed-63afa3cfdc3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:03.318877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8fb76e-ccc0-45c6-89ed-63afa3cfdc3a None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:03.334373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-0c8fb76e-ccc0-45c6-89ed-63afa3cfdc3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:03.334373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:03.334373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:03.334373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:03.334373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:03.334373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:03.334373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:03.334373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:03.334373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:03.334373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:03.334373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:03.334373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:03.334373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:03.334373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:03.334373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:03.335353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:03.336879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:03.336879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:03.336879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:03.336879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:03.336879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 370/727] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:03.336879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fb76e-ccc0-45c6-89ed-63afa3cfdc3a None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:03.350442 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 358/728] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:11:03.350984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729d246-7a47-493d-aab5-cc7f1ba8b550 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:03.426926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:03.461421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729d246-7a47-493d-aab5-cc7f1ba8b550 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:03.463112 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fb76e-ccc0-45c6-89ed-63afa3cfdc3a None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:03.495891 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:03.498255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec60ba-a54a-4b18-a1e0-70e2e5da71d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:03.498982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec60ba-a54a-4b18-a1e0-70e2e5da71d5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:03.499260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec60ba-a54a-4b18-a1e0-70e2e5da71d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:03.499545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec60ba-a54a-4b18-a1e0-70e2e5da71d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:03.500270 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec60ba-a54a-4b18-a1e0-70e2e5da71d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:03.500571 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec60ba-a54a-4b18-a1e0-70e2e5da71d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:03.501989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec60ba-a54a-4b18-a1e0-70e2e5da71d5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:03.503837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a41e8-644a-4926-94c4-63528d34f9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:03.505088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a41e8-644a-4926-94c4-63528d34f9bd None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:03.505512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a41e8-644a-4926-94c4-63528d34f9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:03.505952 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a41e8-644a-4926-94c4-63528d34f9bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:03.514994 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 359/729] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:03.516432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec60ba-a54a-4b18-a1e0-70e2e5da71d5 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:03.519651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a41e8-644a-4926-94c4-63528d34f9bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:03.525116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-848a41e8-644a-4926-94c4-63528d34f9bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:11:03.525116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:03.525116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:03.525116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:03.525116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:03.525116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:03.525116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:03.525116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:03.525116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:03.525116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:03.525116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:03.525116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:03.525116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:03.525116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:03.525116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:03.525116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:03.526365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:03.526365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:03.526365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:03.526365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:03.526365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:03.526365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:03.526365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:03.526365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:03.526365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:03.526365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:03.526365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:03.526365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:03.526365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:03.526365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:03.526365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:03.526365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:03.527247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:03.527247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:03.527247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:03.527247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:03.527247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:03.527247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:03.527247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:03.527247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:03.527247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:03.527247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:03.527247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:11:03.527247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:03.531221 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 371/730] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:11:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:03.532592 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffab549-d69a-4f2d-8e15-a11ccc17148b None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:03.532874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffab549-d69a-4f2d-8e15-a11ccc17148b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:03.533305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffab549-d69a-4f2d-8e15-a11ccc17148b None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:03.537950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 372/731] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:03.537950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffab549-d69a-4f2d-8e15-a11ccc17148b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:03.626669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec60ba-a54a-4b18-a1e0-70e2e5da71d5 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:03.653217 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffab549-d69a-4f2d-8e15-a11ccc17148b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:03.655380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:03.661872 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0dec2-6fa4-488e-85a9-5e3cca5bd4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:03.662523 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0dec2-6fa4-488e-85a9-5e3cca5bd4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:03.662800 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0dec2-6fa4-488e-85a9-5e3cca5bd4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:03.663075 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0dec2-6fa4-488e-85a9-5e3cca5bd4f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:03.673963 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0dec2-6fa4-488e-85a9-5e3cca5bd4f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:03.679358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-1ef0dec2-6fa4-488e-85a9-5e3cca5bd4f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:11:03.679358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:03.679358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:03.679358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:03.679358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:03.679358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:03.679358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:03.679358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:03.679358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:03.679358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:03.679358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:03.679358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:03.679358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:03.679358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:03.679358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:03.679358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:03.680357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:03.680357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:03.680357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:03.680357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:03.680357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:03.680357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:03.680357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:03.680357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:03.680357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:03.680357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:03.680357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:03.680357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:03.680357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:03.680357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:03.680357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:03.680357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:03.681225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:03.681225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:03.681225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:03.681225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:03.681225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:03.681225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:03.681225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:03.681225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:03.681225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:03.681225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:03.681225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:11:03.681225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:03.681225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 360/732] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:11:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:03.686423 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0dec2-6fa4-488e-85a9-5e3cca5bd4f4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:03.693593 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d8196-62df-4055-97e5-2e8797e7111b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:03.694493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d8196-62df-4055-97e5-2e8797e7111b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:03.694992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d8196-62df-4055-97e5-2e8797e7111b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:03.695882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d8196-62df-4055-97e5-2e8797e7111b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:03.706497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d8196-62df-4055-97e5-2e8797e7111b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:03.708209 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d8196-62df-4055-97e5-2e8797e7111b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:03.724964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-6e9d8196-62df-4055-97e5-2e8797e7111b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:11:03.724964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:03.724964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:03.724964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:03.724964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:03.724964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:03.724964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:03.724964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:03.724964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:03.724964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:03.724964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:03.724964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:03.724964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:11:03.724964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:11:03.724964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:11:03.724964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:03.725985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:03.726957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:03.726957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:03.726957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:11:03.726957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:11:03.726957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:11:03.726957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:11:03.726957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:11:03.726957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:11:03.726957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:11:03.726957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:03.726957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 373/733] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 15:11:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:03.728080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e91e2-2d72-4f8d-9bc6-43f4b2762543 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:03.728080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e91e2-2d72-4f8d-9bc6-43f4b2762543 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:03.728249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e91e2-2d72-4f8d-9bc6-43f4b2762543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:03.747403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-7c8e91e2-2d72-4f8d-9bc6-43f4b2762543 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:03.748593 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-7c8e91e2-2d72-4f8d-9bc6-43f4b2762543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:03.797617 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0dec2-6fa4-488e-85a9-5e3cca5bd4f4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:03.802674 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:03.830713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 374/734] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:11:03.839310 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ba1f7-f4cf-4fce-8785-80dff8504aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:03.839942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ba1f7-f4cf-4fce-8785-80dff8504aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:03.840215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ba1f7-f4cf-4fce-8785-80dff8504aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:03.840484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ba1f7-f4cf-4fce-8785-80dff8504aa4 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:03.841207 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ba1f7-f4cf-4fce-8785-80dff8504aa4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:03.841507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ba1f7-f4cf-4fce-8785-80dff8504aa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:03.842713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e91e2-2d72-4f8d-9bc6-43f4b2762543 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:03.843020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ba1f7-f4cf-4fce-8785-80dff8504aa4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:03.856453 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 361/735] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:11:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:03.857705 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ba1f7-f4cf-4fce-8785-80dff8504aa4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:03.925017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:03.955905 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e91e2-2d72-4f8d-9bc6-43f4b2762543 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:03.976422 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ba1f7-f4cf-4fce-8785-80dff8504aa4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:03.993578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c6dc7-6d32-487a-af2c-dfc5c2eca74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:03.994216 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c6dc7-6d32-487a-af2c-dfc5c2eca74b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:03.994499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c6dc7-6d32-487a-af2c-dfc5c2eca74b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:03.994774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c6dc7-6d32-487a-af2c-dfc5c2eca74b None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:03.995501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c6dc7-6d32-487a-af2c-dfc5c2eca74b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:03.995810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c6dc7-6d32-487a-af2c-dfc5c2eca74b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:03.997196 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c6dc7-6d32-487a-af2c-dfc5c2eca74b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:04.005948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 375/736] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:11:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:04.006754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c6dc7-6d32-487a-af2c-dfc5c2eca74b None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:04.019621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1564d2-df57-4f8d-a43d-172f93f7198a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:04.020727 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1564d2-df57-4f8d-a43d-172f93f7198a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:04.021196 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1564d2-df57-4f8d-a43d-172f93f7198a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:04.021538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1564d2-df57-4f8d-a43d-172f93f7198a None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:04.022472 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1564d2-df57-4f8d-a43d-172f93f7198a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:04.022782 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1564d2-df57-4f8d-a43d-172f93f7198a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:04.024479 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1564d2-df57-4f8d-a43d-172f93f7198a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:04.049836 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 362/737] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 15:11:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:04.049836 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1564d2-df57-4f8d-a43d-172f93f7198a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:04.059794 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:04.118812 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c6dc7-6d32-487a-af2c-dfc5c2eca74b None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:04.153305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1564d2-df57-4f8d-a43d-172f93f7198a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:04.155635 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38256a2e-400c-485c-b4ef-cf581a14ed37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:04.156286 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38256a2e-400c-485c-b4ef-cf581a14ed37 None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:04.156610 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38256a2e-400c-485c-b4ef-cf581a14ed37 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:04.156892 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38256a2e-400c-485c-b4ef-cf581a14ed37 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:04.157599 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38256a2e-400c-485c-b4ef-cf581a14ed37 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:04.159026 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38256a2e-400c-485c-b4ef-cf581a14ed37 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:04.165176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:04.185828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 376/738] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 15:11:04 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:11:04.185828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38256a2e-400c-485c-b4ef-cf581a14ed37 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:04.218328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7233fed-eca9-4d7a-b64b-10715dd867e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:04.219023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7233fed-eca9-4d7a-b64b-10715dd867e9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:04.219302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7233fed-eca9-4d7a-b64b-10715dd867e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:04.219571 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7233fed-eca9-4d7a-b64b-10715dd867e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:04.236312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7233fed-eca9-4d7a-b64b-10715dd867e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:04.237872 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7233fed-eca9-4d7a-b64b-10715dd867e9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:04.256438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-a7233fed-eca9-4d7a-b64b-10715dd867e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:04.256438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:04.256438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:04.256438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:04.256438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:04.256438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:04.256438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:04.256438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:04.256438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:04.256438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:04.256438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:04.256438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:04.256438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:04.256438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:04.256438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:04.257431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:04.258415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:04.258415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:04.258415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:04.258415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:04.262257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 363/739] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:04.263250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7233fed-eca9-4d7a-b64b-10715dd867e9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:04.338330 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38256a2e-400c-485c-b4ef-cf581a14ed37 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:04.340711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:04.384853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb979ce-beef-4a89-8674-ba5efdbc9f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:04.386000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb979ce-beef-4a89-8674-ba5efdbc9f0c None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:04.386355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb979ce-beef-4a89-8674-ba5efdbc9f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:04.386735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb979ce-beef-4a89-8674-ba5efdbc9f0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:04.397159 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7233fed-eca9-4d7a-b64b-10715dd867e9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:04.401142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb979ce-beef-4a89-8674-ba5efdbc9f0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:04.402750 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb979ce-beef-4a89-8674-ba5efdbc9f0c None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:04.422496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-ccb979ce-beef-4a89-8674-ba5efdbc9f0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:04.422496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:04.422496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:04.422496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:04.422496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:04.422496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:04.422496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:04.422496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:04.422496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:04.422496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:04.422496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:04.422496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:04.422496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:04.422496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:04.422496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:04.423432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:04.429416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:04.429416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:04.429416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:04.429416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:04.429416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 377/740] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:04.429416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb979ce-beef-4a89-8674-ba5efdbc9f0c None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:04.432577 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb3c8e-00c4-4560-8267-cc231e41e36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:04.433210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb3c8e-00c4-4560-8267-cc231e41e36f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:04.433483 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb3c8e-00c4-4560-8267-cc231e41e36f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:04.433750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb3c8e-00c4-4560-8267-cc231e41e36f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:04.464872 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:04.510501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb3c8e-00c4-4560-8267-cc231e41e36f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:04.512001 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb3c8e-00c4-4560-8267-cc231e41e36f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:04.531978 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb979ce-beef-4a89-8674-ba5efdbc9f0c None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:04.570581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec5036-b0fe-443f-9b61-e4f2c903e51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:04.571207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec5036-b0fe-443f-9b61-e4f2c903e51a None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:04.571479 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec5036-b0fe-443f-9b61-e4f2c903e51a None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:04.571749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec5036-b0fe-443f-9b61-e4f2c903e51a None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:04.572440 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec5036-b0fe-443f-9b61-e4f2c903e51a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:04.572739 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec5036-b0fe-443f-9b61-e4f2c903e51a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:04.574124 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec5036-b0fe-443f-9b61-e4f2c903e51a None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:04.591172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 378/741] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:04.591172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8393a-74a6-4b12-8bef-e989cdc2e514 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:04.591172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8393a-74a6-4b12-8bef-e989cdc2e514 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:04.591172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8393a-74a6-4b12-8bef-e989cdc2e514 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:04.591172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 379/742] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:04.591172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8393a-74a6-4b12-8bef-e989cdc2e514 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:04.609379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:04.631668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 364/743] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:11:04 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:11:04.632539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb3c8e-00c4-4560-8267-cc231e41e36f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:04.695866 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:04.723036 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8393a-74a6-4b12-8bef-e989cdc2e514 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:04.744296 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb3c8e-00c4-4560-8267-cc231e41e36f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:04.762847 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1107f4b-d64b-4bbb-8e13-2d87ec70dd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:04.763469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1107f4b-d64b-4bbb-8e13-2d87ec70dd22 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:04.763718 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1107f4b-d64b-4bbb-8e13-2d87ec70dd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:04.764007 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1107f4b-d64b-4bbb-8e13-2d87ec70dd22 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:04.764865 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1107f4b-d64b-4bbb-8e13-2d87ec70dd22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:04.764985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1107f4b-d64b-4bbb-8e13-2d87ec70dd22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:04.766413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1107f4b-d64b-4bbb-8e13-2d87ec70dd22 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:04.779496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 380/744] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:04.780813 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a5ee2-cfbc-4ddf-a760-710b2799212e None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:04.781098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a5ee2-cfbc-4ddf-a760-710b2799212e None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:04.781430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a5ee2-cfbc-4ddf-a760-710b2799212e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:04.788219 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16204b8-08bc-48be-ab74-b3b1e8cd0fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:04.788883 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16204b8-08bc-48be-ab74-b3b1e8cd0fbe None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:04.789164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16204b8-08bc-48be-ab74-b3b1e8cd0fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:04.789440 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16204b8-08bc-48be-ab74-b3b1e8cd0fbe None admin] PATH_INFO: `/v3/users/swift` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:04.797028 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16204b8-08bc-48be-ab74-b3b1e8cd0fbe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:04.798511 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16204b8-08bc-48be-ab74-b3b1e8cd0fbe None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:04.800936 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-008a5ee2-cfbc-4ddf-a760-710b2799212e None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:04.802178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-008a5ee2-cfbc-4ddf-a760-710b2799212e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:04.810519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e16204b8-08bc-48be-ab74-b3b1e8cd0fbe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:11:04.810519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:04.810519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:04.810519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:04.810519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:04.810519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:04.810519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:04.810519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:04.810519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:04.810519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:04.810519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:04.810519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:04.810519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:11:04.810519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:11:04.810519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:11:04.810519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:04.811490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:04.817851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:04.817851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:04.817851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:11:04.817851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:11:04.817851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:11:04.817851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:11:04.817851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:11:04.817851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:11:04.817851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:11:04.817851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:04.817851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 365/745] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:11:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:04.817851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16204b8-08bc-48be-ab74-b3b1e8cd0fbe None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:04.881904 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 381/746] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:04 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:11:04.894767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a5ee2-cfbc-4ddf-a760-710b2799212e None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:04.902895 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:04.929805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16204b8-08bc-48be-ab74-b3b1e8cd0fbe None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:04.948406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:04.966411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3743a2-070c-4cb5-a549-08ce3bdd0554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:04.966995 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3743a2-070c-4cb5-a549-08ce3bdd0554 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:04.967272 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3743a2-070c-4cb5-a549-08ce3bdd0554 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:04.967724 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3743a2-070c-4cb5-a549-08ce3bdd0554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:04.978290 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3743a2-070c-4cb5-a549-08ce3bdd0554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:04.979966 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3743a2-070c-4cb5-a549-08ce3bdd0554 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:04.994700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-3f3743a2-070c-4cb5-a549-08ce3bdd0554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:04.994700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:04.994700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:04.994700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:04.994700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:04.994700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:04.994700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:04.994700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:04.994700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:04.994700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:04.994700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:04.994700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:04.994700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:04.994700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:04.994700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:04.995761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:04.996660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:04.996660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:04.996660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:04.996660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:04.996660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 366/747] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:04.997494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3743a2-070c-4cb5-a549-08ce3bdd0554 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:05.012569 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a5ee2-cfbc-4ddf-a760-710b2799212e None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:05.050153 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d5494-3012-45c3-8b40-85035183f0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:05.050828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d5494-3012-45c3-8b40-85035183f0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:05.051102 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d5494-3012-45c3-8b40-85035183f0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:05.051369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d5494-3012-45c3-8b40-85035183f0a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:05.058662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d5494-3012-45c3-8b40-85035183f0a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:05.060122 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d5494-3012-45c3-8b40-85035183f0a2 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:05.071634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-8b2d5494-3012-45c3-8b40-85035183f0a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:11:05.071634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:05.071634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:05.071634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:05.071634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:05.071634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:05.071634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:05.071634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:05.071634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:05.071634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:05.071634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:05.071634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:05.071634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:11:05.071634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:11:05.071634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:11:05.071634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:05.072576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:05.073577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:05.073577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:05.073577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:11:05.073577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:11:05.073577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:11:05.073577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:11:05.073577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:11:05.073577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:11:05.073577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:11:05.073577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:05.073577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 382/748] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 15:11:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:05.074227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d5494-3012-45c3-8b40-85035183f0a2 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:05.113052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3743a2-070c-4cb5-a549-08ce3bdd0554 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:05.143509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:05.148179 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688a2ad-6242-4dad-b6e6-37e0d5d657a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:05.149360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688a2ad-6242-4dad-b6e6-37e0d5d657a5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:05.149360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688a2ad-6242-4dad-b6e6-37e0d5d657a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:05.149360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688a2ad-6242-4dad-b6e6-37e0d5d657a5 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:05.150151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688a2ad-6242-4dad-b6e6-37e0d5d657a5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:05.150515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688a2ad-6242-4dad-b6e6-37e0d5d657a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:05.151956 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688a2ad-6242-4dad-b6e6-37e0d5d657a5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:05.175637 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 367/749] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 15:11:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:05.175637 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688a2ad-6242-4dad-b6e6-37e0d5d657a5 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:05.187068 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d5494-3012-45c3-8b40-85035183f0a2 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:05.204624 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:05.227939 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade2c1d-134c-4263-abc1-933c6d049cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:05.228687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade2c1d-134c-4263-abc1-933c6d049cb5 None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:05.228992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade2c1d-134c-4263-abc1-933c6d049cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:05.229409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade2c1d-134c-4263-abc1-933c6d049cb5 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:05.230232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ade2c1d-134c-4263-abc1-933c6d049cb5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:05.231687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ade2c1d-134c-4263-abc1-933c6d049cb5 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:05.257816 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 383/750] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 15:11:05 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:11:05.257816 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade2c1d-134c-4263-abc1-933c6d049cb5 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:05.318320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688a2ad-6242-4dad-b6e6-37e0d5d657a5 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:05.378696 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e00c0a-c0de-4d34-86de-5cb845d82e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:05.379355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e00c0a-c0de-4d34-86de-5cb845d82e7e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:05.379761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e00c0a-c0de-4d34-86de-5cb845d82e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:05.380147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e00c0a-c0de-4d34-86de-5cb845d82e7e None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:05.380862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e00c0a-c0de-4d34-86de-5cb845d82e7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:05.381415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e00c0a-c0de-4d34-86de-5cb845d82e7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:05.383666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e00c0a-c0de-4d34-86de-5cb845d82e7e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:05.388957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade2c1d-134c-4263-abc1-933c6d049cb5 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:05.400413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 368/751] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:05.402084 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e00c0a-c0de-4d34-86de-5cb845d82e7e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:05.418058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:05.433422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb6371-6c09-4c52-b97b-199b47fd8c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:05.434304 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb6371-6c09-4c52-b97b-199b47fd8c69 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:05.434577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb6371-6c09-4c52-b97b-199b47fd8c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:05.434842 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb6371-6c09-4c52-b97b-199b47fd8c69 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:05.435670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb6371-6c09-4c52-b97b-199b47fd8c69 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:05.435972 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb6371-6c09-4c52-b97b-199b47fd8c69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:05.437420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb6371-6c09-4c52-b97b-199b47fd8c69 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:05.464035 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 384/752] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 15:11:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:05.465214 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb6371-6c09-4c52-b97b-199b47fd8c69 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:05.518636 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:05.541772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e00c0a-c0de-4d34-86de-5cb845d82e7e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:05.576042 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb6371-6c09-4c52-b97b-199b47fd8c69 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:05.576657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de00261-37b8-4237-8243-4a054672476b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:05.577289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de00261-37b8-4237-8243-4a054672476b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:05.577562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de00261-37b8-4237-8243-4a054672476b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:05.577854 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de00261-37b8-4237-8243-4a054672476b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:05.612854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b71ab-c678-4c03-a0db-0daf1248be44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:05.613928 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b71ab-c678-4c03-a0db-0daf1248be44 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:05.614440 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b71ab-c678-4c03-a0db-0daf1248be44 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:05.614939 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b71ab-c678-4c03-a0db-0daf1248be44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:05.622362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b71ab-c678-4c03-a0db-0daf1248be44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:05.624174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b71ab-c678-4c03-a0db-0daf1248be44 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:05.636291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-584b71ab-c678-4c03-a0db-0daf1248be44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:05.636291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:05.636291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:05.636291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:05.636291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:05.636291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:05.636291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:05.636291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:05.636291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:05.636291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:05.636291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:05.636291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:05.636291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:11:05.636291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:11:05.636291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:05.637244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:05.638204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:11:05.638204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:11:05.638204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:11:05.638204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:11:05.638204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:11:05.638204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:11:05.638204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:05.638204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:05.640703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 385/753] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:05.646308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b71ab-c678-4c03-a0db-0daf1248be44 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:05.648509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de00261-37b8-4237-8243-4a054672476b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:05.650100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de00261-37b8-4237-8243-4a054672476b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:05.692894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:05.751985 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:05.760277 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b71ab-c678-4c03-a0db-0daf1248be44 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:05.777255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 369/754] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:11:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:11:05.778323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de00261-37b8-4237-8243-4a054672476b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:05.797706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d61e1-c99b-4eb0-ad88-223e4f6d3ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:05.798812 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d61e1-c99b-4eb0-ad88-223e4f6d3ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:05.799293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d61e1-c99b-4eb0-ad88-223e4f6d3ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:05.799757 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d61e1-c99b-4eb0-ad88-223e4f6d3ef9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:05.811026 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d61e1-c99b-4eb0-ad88-223e4f6d3ef9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:05.812495 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d61e1-c99b-4eb0-ad88-223e4f6d3ef9 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:05.830098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-ad3d61e1-c99b-4eb0-ad88-223e4f6d3ef9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:05.830098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:05.830098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:05.830098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:05.830098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:05.830098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:05.830098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:05.830098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:05.830098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:05.830098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:05.830098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:05.830098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:05.830098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:05.830098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:05.830098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:05.831074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:05.831970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:05.831970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:05.831970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:05.831970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:05.831970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 386/755] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:05.832690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d61e1-c99b-4eb0-ad88-223e4f6d3ef9 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:05.893667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de00261-37b8-4237-8243-4a054672476b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:05.929251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:05.938030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d61e1-c99b-4eb0-ad88-223e4f6d3ef9 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:05.944501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc003e-7598-42ae-a27a-84797bc39e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:05.945446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc003e-7598-42ae-a27a-84797bc39e76 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:05.946521 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc003e-7598-42ae-a27a-84797bc39e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:05.947023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc003e-7598-42ae-a27a-84797bc39e76 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:05.948449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc003e-7598-42ae-a27a-84797bc39e76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:05.949111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc003e-7598-42ae-a27a-84797bc39e76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:05.951487 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc003e-7598-42ae-a27a-84797bc39e76 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:05.968832 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 370/756] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:11:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:05.970117 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc003e-7598-42ae-a27a-84797bc39e76 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:05.980336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ec478-f458-43d7-bff7-b945c2138dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:05.981334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ec478-f458-43d7-bff7-b945c2138dfd None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:05.981334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ec478-f458-43d7-bff7-b945c2138dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:05.981508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ec478-f458-43d7-bff7-b945c2138dfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:06.002258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ec478-f458-43d7-bff7-b945c2138dfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:06.003893 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ec478-f458-43d7-bff7-b945c2138dfd None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:06.025383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-7a3ec478-f458-43d7-bff7-b945c2138dfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:06.025383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:06.025383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:06.025383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:06.025383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:06.025383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:06.025383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:06.025383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:06.025383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:06.025383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:06.025383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:06.025383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:06.025383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:06.025383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:06.025383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:06.026402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:06.027322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:06.027322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:06.027322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:06.027322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:06.032854 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:06.033431 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 387/757] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:06.036592 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ec478-f458-43d7-bff7-b945c2138dfd None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:06.093871 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc003e-7598-42ae-a27a-84797bc39e76 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:06.130076 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd288348-447f-48b7-bc24-606ae477eaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:06.130859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd288348-447f-48b7-bc24-606ae477eaaa None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:06.131240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd288348-447f-48b7-bc24-606ae477eaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:06.131615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd288348-447f-48b7-bc24-606ae477eaaa None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:06.132422 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd288348-447f-48b7-bc24-606ae477eaaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:06.132823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd288348-447f-48b7-bc24-606ae477eaaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:06.134380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd288348-447f-48b7-bc24-606ae477eaaa None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:06.148494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 371/758] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:06.149607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd288348-447f-48b7-bc24-606ae477eaaa None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:06.168822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ec478-f458-43d7-bff7-b945c2138dfd None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:06.212938 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a2fbc-f1f3-457b-95d5-29ebeb23b971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:06.214166 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a2fbc-f1f3-457b-95d5-29ebeb23b971 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:06.214620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a2fbc-f1f3-457b-95d5-29ebeb23b971 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:06.215303 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a2fbc-f1f3-457b-95d5-29ebeb23b971 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:06.216262 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a2fbc-f1f3-457b-95d5-29ebeb23b971 None admin] RBAC: Adding query filter params (role.id=624a5f7ed1cf440cb177291e89bb11ff, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=38b18e5d1e8843c6a08b1f5609cc60d5) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:06.216861 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a2fbc-f1f3-457b-95d5-29ebeb23b971 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:06.219142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a2fbc-f1f3-457b-95d5-29ebeb23b971 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:06.226678 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 388/759] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:11:06 2022] GET /identity/v3/role_assignments?user.id=38b18e5d1e8843c6a08b1f5609cc60d5&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=624a5f7ed1cf440cb177291e89bb11ff => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:06.227515 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a2fbc-f1f3-457b-95d5-29ebeb23b971 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:06.245154 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:06.263783 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd288348-447f-48b7-bc24-606ae477eaaa None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:06.266737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:06.306493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddfacf-b75d-4194-806b-7fc72f972781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:06.307352 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddfacf-b75d-4194-806b-7fc72f972781 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:06.307791 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddfacf-b75d-4194-806b-7fc72f972781 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:06.308263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddfacf-b75d-4194-806b-7fc72f972781 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:06.309072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ddfacf-b75d-4194-806b-7fc72f972781 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:06.309382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ddfacf-b75d-4194-806b-7fc72f972781 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:06.311078 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ddfacf-b75d-4194-806b-7fc72f972781 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:06.329285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 372/760] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:06.329285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f737601-8b14-4664-bddf-c740b8943d5a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:06.329285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f737601-8b14-4664-bddf-c740b8943d5a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:06.329285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f737601-8b14-4664-bddf-c740b8943d5a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:06.329285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 373/761] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:06.330379 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f737601-8b14-4664-bddf-c740b8943d5a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:06.358233 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a2fbc-f1f3-457b-95d5-29ebeb23b971 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:06.402744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edb36f-6c90-49e1-a471-85fce79e15c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:06.403443 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edb36f-6c90-49e1-a471-85fce79e15c3 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:06.403769 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edb36f-6c90-49e1-a471-85fce79e15c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:06.404046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edb36f-6c90-49e1-a471-85fce79e15c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:06.414434 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99edb36f-6c90-49e1-a471-85fce79e15c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:06.416108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99edb36f-6c90-49e1-a471-85fce79e15c3 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:06.433107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-99edb36f-6c90-49e1-a471-85fce79e15c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:06.433107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:06.433107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:06.433107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:06.433107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:06.433107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:06.433107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:06.433107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:06.433107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:06.433107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:06.433107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:06.433107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:06.433107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:11:06.433107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:11:06.433107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:06.434132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:06.438741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:11:06.438741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:11:06.438741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:11:06.438741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:11:06.438741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:11:06.438741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:11:06.438741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:06.438741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:06.438741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 389/762] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:06.438741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75d13f-1103-4c50-9d49-86ad3da85586 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:06.438741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75d13f-1103-4c50-9d49-86ad3da85586 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:06.439342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75d13f-1103-4c50-9d49-86ad3da85586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:06.465409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f737601-8b14-4664-bddf-c740b8943d5a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:06.465633 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-1a75d13f-1103-4c50-9d49-86ad3da85586 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:06.467382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1a75d13f-1103-4c50-9d49-86ad3da85586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:06.505652 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20365b2e-b776-4d0a-a322-39923061da7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:06.506437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20365b2e-b776-4d0a-a322-39923061da7e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:06.506718 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20365b2e-b776-4d0a-a322-39923061da7e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:06.506993 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20365b2e-b776-4d0a-a322-39923061da7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:06.516419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20365b2e-b776-4d0a-a322-39923061da7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:06.517907 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20365b2e-b776-4d0a-a322-39923061da7e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:06.531416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-20365b2e-b776-4d0a-a322-39923061da7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:06.531416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:06.531416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:06.531416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:06.531416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:06.531416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:06.531416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:06.531416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:06.531416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:06.531416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:06.531416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:06.531416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:06.531416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:11:06.531416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:11:06.531416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:06.532353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:06.533862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:11:06.533862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:11:06.533862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:11:06.533862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:11:06.533862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:11:06.533862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:11:06.533862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:06.533862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:06.533862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 374/763] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:06.533862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20365b2e-b776-4d0a-a322-39923061da7e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:06.552518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:06.566298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 390/764] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:06.566298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75d13f-1103-4c50-9d49-86ad3da85586 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:06.572262 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:06.681512 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20365b2e-b776-4d0a-a322-39923061da7e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:06.699400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75d13f-1103-4c50-9d49-86ad3da85586 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:06.762118 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb17dd3-8960-4dd4-9aff-1fd8eb4142a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:06.762118 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb17dd3-8960-4dd4-9aff-1fd8eb4142a0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:06.762118 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb17dd3-8960-4dd4-9aff-1fd8eb4142a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:06.762118 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb17dd3-8960-4dd4-9aff-1fd8eb4142a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:06.762674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb17dd3-8960-4dd4-9aff-1fd8eb4142a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:06.762674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb17dd3-8960-4dd4-9aff-1fd8eb4142a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:06.763445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ee6f6-2a74-4133-a03f-69a1b1df5d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:06.764291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ee6f6-2a74-4133-a03f-69a1b1df5d0d None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:06.764570 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ee6f6-2a74-4133-a03f-69a1b1df5d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:06.764842 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ee6f6-2a74-4133-a03f-69a1b1df5d0d None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:06.765668 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ee6f6-2a74-4133-a03f-69a1b1df5d0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:06.766066 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb17dd3-8960-4dd4-9aff-1fd8eb4142a0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:06.766744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ee6f6-2a74-4133-a03f-69a1b1df5d0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:06.768215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ee6f6-2a74-4133-a03f-69a1b1df5d0d None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:06.785768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 375/765] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:11:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:06.787002 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb17dd3-8960-4dd4-9aff-1fd8eb4142a0 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:06.806211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 391/766] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:11:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:06.807115 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ee6f6-2a74-4133-a03f-69a1b1df5d0d None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:06.890061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:06.909280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb17dd3-8960-4dd4-9aff-1fd8eb4142a0 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:06.953355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ce4d5-15de-42cf-9148-89e8c2e6572d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:06.954554 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ce4d5-15de-42cf-9148-89e8c2e6572d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:06.955104 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ce4d5-15de-42cf-9148-89e8c2e6572d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:06.955565 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ce4d5-15de-42cf-9148-89e8c2e6572d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:06.956445 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ce4d5-15de-42cf-9148-89e8c2e6572d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:06.958098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ce4d5-15de-42cf-9148-89e8c2e6572d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:06.971109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:06.978010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-989ce4d5-15de-42cf-9148-89e8c2e6572d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:11:06.978010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:06.978010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:06.978010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:06.978010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:06.978010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:06.978010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:06.978010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:06.978010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:11:06.978010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:06.978010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:06.978010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:06.978010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:06.978010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:06.978010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:06.978010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:06.979255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:06.979255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:06.979255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:06.979255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:06.979255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:06.979255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:06.979255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:06.979255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:06.979255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:06.979255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:06.979255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:06.979255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:06.979255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:06.979255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:06.979255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:06.979255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:06.980090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:06.980090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:06.980090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:06.980090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:06.980090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:06.980090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:06.980090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:06.980090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:06.980090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:06.980090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:06.980090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:06.980090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:06.980090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:06.980090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:11:06.980090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:06.980090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 376/767] 10.222.0.48 () {56 vars in 1129 bytes} [Mon Aug 1 15:11:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:06.989240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ce4d5-15de-42cf-9148-89e8c2e6572d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:07.036233 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ee6f6-2a74-4133-a03f-69a1b1df5d0d None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:07.111322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4a453-c012-43e4-804b-e934762b2f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:07.112035 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4a453-c012-43e4-804b-e934762b2f03 None admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:07.112310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4a453-c012-43e4-804b-e934762b2f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:07.112586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4a453-c012-43e4-804b-e934762b2f03 None admin] PATH_INFO: `/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/2de0ff99a2b74979a28a8b10e9ebdfcd/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:07.118259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4a453-c012-43e4-804b-e934762b2f03 None admin] RBAC: Authorizing `identity:create_grant(project_id=218a9ee5cb2842248b6995d2562bb74b, user_id=2de0ff99a2b74979a28a8b10e9ebdfcd, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:07.119722 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4a453-c012-43e4-804b-e934762b2f03 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:07.145902 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ce4d5-15de-42cf-9148-89e8c2e6572d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:07.165817 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 392/768] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:11:06 2022] PUT /identity/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/2de0ff99a2b74979a28a8b10e9ebdfcd/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:11:07.165817 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4a453-c012-43e4-804b-e934762b2f03 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:07.224575 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da18b1c-1c4d-43ca-9fde-7367273e469d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:07.225629 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da18b1c-1c4d-43ca-9fde-7367273e469d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:07.226150 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da18b1c-1c4d-43ca-9fde-7367273e469d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:07.226614 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da18b1c-1c4d-43ca-9fde-7367273e469d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:07.227726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da18b1c-1c4d-43ca-9fde-7367273e469d None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=25a26f54accf4264aef646c3757de7e9) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:07.228252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da18b1c-1c4d-43ca-9fde-7367273e469d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:07.230312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da18b1c-1c4d-43ca-9fde-7367273e469d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:07.238142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 377/769] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:11:06 2022] GET /identity/v3/role_assignments?user.id=25a26f54accf4264aef646c3757de7e9&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:07.342048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4a453-c012-43e4-804b-e934762b2f03 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:07.366791 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:07.385057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a154b-b72a-4553-a48d-baf8c9225b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:07.385710 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a154b-b72a-4553-a48d-baf8c9225b12 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:07.386040 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a154b-b72a-4553-a48d-baf8c9225b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:07.386318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a154b-b72a-4553-a48d-baf8c9225b12 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:07.387029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a154b-b72a-4553-a48d-baf8c9225b12 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:07.387406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a154b-b72a-4553-a48d-baf8c9225b12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:07.388830 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a154b-b72a-4553-a48d-baf8c9225b12 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:07.394898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 393/770] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 15:11:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:07.401300 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da18b1c-1c4d-43ca-9fde-7367273e469d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:07.420826 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:07.516069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da18b1c-1c4d-43ca-9fde-7367273e469d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:07.541420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59677175-2f47-496c-b53d-62631af24a16 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:07.541802 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59677175-2f47-496c-b53d-62631af24a16 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:07.542001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59677175-2f47-496c-b53d-62631af24a16 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:07.543537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 394/771] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:07.548852 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a394bc1-1b20-4141-bbeb-2598d64afda2 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:07.549367 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a394bc1-1b20-4141-bbeb-2598d64afda2 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:07.549926 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a394bc1-1b20-4141-bbeb-2598d64afda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:07.555265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa64e0-fad0-4fd4-935a-bf6e615418bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:07.555885 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa64e0-fad0-4fd4-935a-bf6e615418bb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:07.556166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa64e0-fad0-4fd4-935a-bf6e615418bb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:07.556442 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa64e0-fad0-4fd4-935a-bf6e615418bb None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:07.557135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa64e0-fad0-4fd4-935a-bf6e615418bb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:07.557448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa64e0-fad0-4fd4-935a-bf6e615418bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:07.560280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa64e0-fad0-4fd4-935a-bf6e615418bb None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:07.566579 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 378/772] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 15:11:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:07.571725 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-7a394bc1-1b20-4141-bbeb-2598d64afda2 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:07.573183 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-7a394bc1-1b20-4141-bbeb-2598d64afda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:07.574681 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa64e0-fad0-4fd4-935a-bf6e615418bb None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:07.662250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:07.686353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa64e0-fad0-4fd4-935a-bf6e615418bb None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:07.724982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc094298-cf00-4122-8d55-2f91c865438d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:07.725626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc094298-cf00-4122-8d55-2f91c865438d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:07.725941 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc094298-cf00-4122-8d55-2f91c865438d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:07.726220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc094298-cf00-4122-8d55-2f91c865438d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:07.726993 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc094298-cf00-4122-8d55-2f91c865438d None admin] RBAC: Adding query filter params (interface=public, service_id=ef36cfe3373f4130bb93084536cdb315, region_id=RegionOne) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:07.727299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc094298-cf00-4122-8d55-2f91c865438d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:07.728686 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc094298-cf00-4122-8d55-2f91c865438d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:07.737620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 379/773] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 15:11:07 2022] GET /identity/v3/endpoints?service_id=ef36cfe3373f4130bb93084536cdb315&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:07.742706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc094298-cf00-4122-8d55-2f91c865438d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:07.773970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 395/774] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:11:07.810353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a394bc1-1b20-4141-bbeb-2598d64afda2 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:07.852370 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc094298-cf00-4122-8d55-2f91c865438d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:07.914985 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9862d-5be4-4c3e-afc2-75dc2f09859f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:07.915539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9862d-5be4-4c3e-afc2-75dc2f09859f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:07.915816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9862d-5be4-4c3e-afc2-75dc2f09859f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:07.916091 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9862d-5be4-4c3e-afc2-75dc2f09859f None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:07.916749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9862d-5be4-4c3e-afc2-75dc2f09859f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:07.918159 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9862d-5be4-4c3e-afc2-75dc2f09859f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:07.938855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 380/775] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 15:11:07 2022] GET /identity/v3/services => generated 772 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:08.002147 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a394bc1-1b20-4141-bbeb-2598d64afda2 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:08.024874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:08.051443 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11c93d-7584-487d-8f91-d387d3203afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:08.052123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11c93d-7584-487d-8f91-d387d3203afe None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:08.052396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11c93d-7584-487d-8f91-d387d3203afe None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:08.052665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11c93d-7584-487d-8f91-d387d3203afe None admin] PATH_INFO: `/v3/services/network` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:08.053306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11c93d-7584-487d-8f91-d387d3203afe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:08.054708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11c93d-7584-487d-8f91-d387d3203afe None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-ee11c93d-7584-487d-8f91-d387d3203afe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:08.067106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:08.068176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:08.068176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:08.068176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:08.068176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:08.068176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:08.068176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:08.068176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:08.068176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:08.068176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:08.068176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:08.068176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:08.068176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:08.068176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:08.068176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:08.068176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:08.068176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:08.069028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:08.069028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:08.069028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:08.069028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:08.069028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:08.069028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:08.069028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:08.069028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:08.069028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:08.069028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:08.069028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:08.069028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:08.069028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:11:08.069028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:08.069028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 396/776] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:07 2022] GET /identity/v3/services/network => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:08.076666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9862d-5be4-4c3e-afc2-75dc2f09859f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:08.080942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:08.194612 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9862d-5be4-4c3e-afc2-75dc2f09859f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:08.231359 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0bc29-de9f-4c22-b26f-a7682b820a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:08.232002 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0bc29-de9f-4c22-b26f-a7682b820a32 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:08.232638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0bc29-de9f-4c22-b26f-a7682b820a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:08.232945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0bc29-de9f-4c22-b26f-a7682b820a32 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:08.233635 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0bc29-de9f-4c22-b26f-a7682b820a32 None admin] RBAC: Adding query filter params (name=network) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:08.234030 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0bc29-de9f-4c22-b26f-a7682b820a32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:08.235528 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0bc29-de9f-4c22-b26f-a7682b820a32 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:08.240950 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 381/777] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 15:11:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:08.246633 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11c93d-7584-487d-8f91-d387d3203afe None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:08.357491 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11c93d-7584-487d-8f91-d387d3203afe None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:08.392014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ba121-a1bd-4a17-b7b0-5f93fa2f769a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:08.392646 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ba121-a1bd-4a17-b7b0-5f93fa2f769a None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:08.392921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ba121-a1bd-4a17-b7b0-5f93fa2f769a None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:08.393190 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ba121-a1bd-4a17-b7b0-5f93fa2f769a None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:08.393898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ba121-a1bd-4a17-b7b0-5f93fa2f769a None admin] RBAC: Adding query filter params (type=network) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:08.394199 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ba121-a1bd-4a17-b7b0-5f93fa2f769a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:08.395572 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ba121-a1bd-4a17-b7b0-5f93fa2f769a None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:08.401058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 397/778] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 15:11:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:08.406817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0bc29-de9f-4c22-b26f-a7682b820a32 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:08.477859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:08.504508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae3636-559b-4bcc-83db-f7480eb75e86 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:08.504829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae3636-559b-4bcc-83db-f7480eb75e86 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:08.505055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae3636-559b-4bcc-83db-f7480eb75e86 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:08.507793 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 398/779] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:11:08.513478 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcaeca-5469-4156-a1e7-b333ce3dd40c None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:08.513768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcaeca-5469-4156-a1e7-b333ce3dd40c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:08.514046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dcaeca-5469-4156-a1e7-b333ce3dd40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:08.518071 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0bc29-de9f-4c22-b26f-a7682b820a32 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:08.555993 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c7afd-de67-4861-b20b-3cfb5ed1bd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:08.556721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c7afd-de67-4861-b20b-3cfb5ed1bd20 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:08.557270 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c7afd-de67-4861-b20b-3cfb5ed1bd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:08.557613 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950c7afd-de67-4861-b20b-3cfb5ed1bd20 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:08.558491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c7afd-de67-4861-b20b-3cfb5ed1bd20 None admin] RBAC: Adding query filter params (interface=public, service_id=20385af4002342ddaa63e9d57956b0cb, region_id=RegionOne) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:08.558868 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c7afd-de67-4861-b20b-3cfb5ed1bd20 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:08.560360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950c7afd-de67-4861-b20b-3cfb5ed1bd20 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:08.562895 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-97dcaeca-5469-4156-a1e7-b333ce3dd40c None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:08.564114 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-97dcaeca-5469-4156-a1e7-b333ce3dd40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:08.566758 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 382/780] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 15:11:08 2022] GET /identity/v3/endpoints?service_id=20385af4002342ddaa63e9d57956b0cb&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:08.571995 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c7afd-de67-4861-b20b-3cfb5ed1bd20 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:08.680307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:08.693207 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950c7afd-de67-4861-b20b-3cfb5ed1bd20 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:08.727363 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:08.735660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f891d-bcd5-4b23-812b-11d83829b80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:08.736562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f891d-bcd5-4b23-812b-11d83829b80b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:08.737020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f891d-bcd5-4b23-812b-11d83829b80b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:08.737472 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f891d-bcd5-4b23-812b-11d83829b80b None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:08.738362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f891d-bcd5-4b23-812b-11d83829b80b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:08.740455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f891d-bcd5-4b23-812b-11d83829b80b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:08.746685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 383/781] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 15:11:08 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:08.785441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 399/782] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:08.825606 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f891d-bcd5-4b23-812b-11d83829b80b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:08.939479 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f891d-bcd5-4b23-812b-11d83829b80b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:08.975705 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813fe0c-07ce-43ba-83e1-7f185ff3c1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:08.976733 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813fe0c-07ce-43ba-83e1-7f185ff3c1da None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:08.977212 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813fe0c-07ce-43ba-83e1-7f185ff3c1da None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:08.977707 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813fe0c-07ce-43ba-83e1-7f185ff3c1da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:08.990445 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7813fe0c-07ce-43ba-83e1-7f185ff3c1da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:08.996340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-7813fe0c-07ce-43ba-83e1-7f185ff3c1da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:11:08.996340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:08.996340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:08.996340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:08.996340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:08.996340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:08.996340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:08.996340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:08.996340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:08.996340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:08.996340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:08.996340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:08.996340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:08.996340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:08.996340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:08.996340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:08.997339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:08.997339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:08.997339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:08.997339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:08.997339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:08.997339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:08.997339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:08.997339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:08.997339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:08.997339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:08.997339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:08.997339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:08.997339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:08.997339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:08.997339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:08.997339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:08.998197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:08.998197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:08.998197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:08.998197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:08.998197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:08.998197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:08.998197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:08.998197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:08.998197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:08.998197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:08.998197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:11:08.998197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:09.001221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 384/783] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:11:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:09.006983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcaeca-5469-4156-a1e7-b333ce3dd40c None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:09.120049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dcaeca-5469-4156-a1e7-b333ce3dd40c None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:09.155246 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f852f2-7ca9-4fc8-b431-c7b620ba7817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:09.155881 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f852f2-7ca9-4fc8-b431-c7b620ba7817 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:09.156206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f852f2-7ca9-4fc8-b431-c7b620ba7817 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:09.156487 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f852f2-7ca9-4fc8-b431-c7b620ba7817 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:09.157228 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f852f2-7ca9-4fc8-b431-c7b620ba7817 None admin] RBAC: Adding query filter params (name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:09.157540 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f852f2-7ca9-4fc8-b431-c7b620ba7817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:09.158959 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f852f2-7ca9-4fc8-b431-c7b620ba7817 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:09.167930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 400/784] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:11:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:09.173221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813fe0c-07ce-43ba-83e1-7f185ff3c1da None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:09.249130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:09.305431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813fe0c-07ce-43ba-83e1-7f185ff3c1da None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:09.345360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cd61b-05f5-4175-ad97-8e7596cc0668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:09.346328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cd61b-05f5-4175-ad97-8e7596cc0668 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:09.346746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cd61b-05f5-4175-ad97-8e7596cc0668 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:09.347183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cd61b-05f5-4175-ad97-8e7596cc0668 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:09.365221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9cd61b-05f5-4175-ad97-8e7596cc0668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:09.371035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9cd61b-05f5-4175-ad97-8e7596cc0668 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:09.386461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-dc9cd61b-05f5-4175-ad97-8e7596cc0668 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:09.386461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:09.386461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:09.386461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:09.386461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:09.386461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:09.386461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:09.386461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:09.386461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:09.386461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:09.386461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:09.386461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:09.386461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:09.386461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:09.386461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:09.387550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:09.388667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:09.388667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:09.388667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:09.388667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:09.394684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 385/785] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:09.399969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f852f2-7ca9-4fc8-b431-c7b620ba7817 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:09.457383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:09.550175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f852f2-7ca9-4fc8-b431-c7b620ba7817 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:09.594837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8858c-4e82-4a95-8383-0ac80e8409b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:09.595474 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8858c-4e82-4a95-8383-0ac80e8409b2 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:09.595751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8858c-4e82-4a95-8383-0ac80e8409b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:09.596026 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8858c-4e82-4a95-8383-0ac80e8409b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:09.596736 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8858c-4e82-4a95-8383-0ac80e8409b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:09.597042 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8858c-4e82-4a95-8383-0ac80e8409b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:09.598471 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8858c-4e82-4a95-8383-0ac80e8409b2 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:09.611833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 401/786] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:09.617058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cd61b-05f5-4175-ad97-8e7596cc0668 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:09.702751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34206c36-a3d1-454f-a04d-d39f2ae319f3 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:09.703279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34206c36-a3d1-454f-a04d-d39f2ae319f3 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:09.703477 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34206c36-a3d1-454f-a04d-d39f2ae319f3 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:09.705137 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 402/787] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:09.710101 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c672b-0ecf-46c4-95c6-2524c49a8bd9 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:09.710385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c672b-0ecf-46c4-95c6-2524c49a8bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:09.710661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c672b-0ecf-46c4-95c6-2524c49a8bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:09.751298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cd61b-05f5-4175-ad97-8e7596cc0668 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:09.757650 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-8a7c672b-0ecf-46c4-95c6-2524c49a8bd9 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:09.758951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8a7c672b-0ecf-46c4-95c6-2524c49a8bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:09.766129 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:09.789024 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f0df9-b2b8-4d64-88ff-1992ade1e602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:09.790233 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f0df9-b2b8-4d64-88ff-1992ade1e602 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:09.790706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f0df9-b2b8-4d64-88ff-1992ade1e602 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:09.791405 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f0df9-b2b8-4d64-88ff-1992ade1e602 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:09.802834 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f0df9-b2b8-4d64-88ff-1992ade1e602 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:09.804944 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f0df9-b2b8-4d64-88ff-1992ade1e602 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:09.820585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-369f0df9-b2b8-4d64-88ff-1992ade1e602 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:11:09.820585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:09.820585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:09.820585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:09.820585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:09.820585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:09.820585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:09.820585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:09.820585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:09.820585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:09.820585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:09.820585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:09.820585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:11:09.820585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:11:09.820585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:11:09.820585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:09.821564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:09.822778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:09.822778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:09.822778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:11:09.822778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:11:09.822778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:11:09.822778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:11:09.822778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:11:09.822778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:11:09.822778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:11:09.822778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:09.824008 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 386/788] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 15:11:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:09.825000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82daf267-45e4-4491-95ef-505768020c13 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:09.825283 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82daf267-45e4-4491-95ef-505768020c13 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:09.825566 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82daf267-45e4-4491-95ef-505768020c13 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:09.827543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 387/789] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:09.832813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82daf267-45e4-4491-95ef-505768020c13 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:09.883874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 403/790] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:09.886398 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596c81-39f3-4e5f-a261-b6f3fe7d3157 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:09.887156 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596c81-39f3-4e5f-a261-b6f3fe7d3157 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:09.887623 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e596c81-39f3-4e5f-a261-b6f3fe7d3157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:09.953352 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82daf267-45e4-4491-95ef-505768020c13 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:09.998931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd74f31-670d-4cdd-a042-d872b2e5c093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:10.000108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd74f31-670d-4cdd-a042-d872b2e5c093 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:10.000593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd74f31-670d-4cdd-a042-d872b2e5c093 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:10.001089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd74f31-670d-4cdd-a042-d872b2e5c093 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:10.002163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd74f31-670d-4cdd-a042-d872b2e5c093 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:10.002672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd74f31-670d-4cdd-a042-d872b2e5c093 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:10.004450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd74f31-670d-4cdd-a042-d872b2e5c093 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:10.027252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 388/791] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 15:11:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:10.028540 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd74f31-670d-4cdd-a042-d872b2e5c093 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:10.049450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-5e596c81-39f3-4e5f-a261-b6f3fe7d3157 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:10.051090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5e596c81-39f3-4e5f-a261-b6f3fe7d3157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:10.060313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:10.074422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:10.137767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 404/792] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:10.139101 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e596c81-39f3-4e5f-a261-b6f3fe7d3157 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:10.265951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e596c81-39f3-4e5f-a261-b6f3fe7d3157 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:10.297450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd74f31-670d-4cdd-a042-d872b2e5c093 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:10.313406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b1f7f-f93b-4a53-9742-0b9d89fa05b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:10.314794 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b1f7f-f93b-4a53-9742-0b9d89fa05b0 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:10.315276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b1f7f-f93b-4a53-9742-0b9d89fa05b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:10.316002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b1f7f-f93b-4a53-9742-0b9d89fa05b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:10.332113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b1f7f-f93b-4a53-9742-0b9d89fa05b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:10.334209 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b1f7f-f93b-4a53-9742-0b9d89fa05b0 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:10.336719 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be959eb1-a3f2-4823-8b1b-fc181ce6403f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:10.337477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be959eb1-a3f2-4823-8b1b-fc181ce6403f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:10.337759 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be959eb1-a3f2-4823-8b1b-fc181ce6403f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:10.338210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be959eb1-a3f2-4823-8b1b-fc181ce6403f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:10.350281 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be959eb1-a3f2-4823-8b1b-fc181ce6403f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:10.352981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-cd1b1f7f-f93b-4a53-9742-0b9d89fa05b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:10.352981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:10.352981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:10.352981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:10.352981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:10.352981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:10.352981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:10.352981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:10.352981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:10.352981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:10.352981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:10.352981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:10.352981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:10.352981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:10.352981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:10.354297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:10.355188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:10.355188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:10.355188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:10.355188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:10.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-be959eb1-a3f2-4823-8b1b-fc181ce6403f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:11:10.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:10.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:10.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:10.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:10.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:10.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:10.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:10.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:10.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:10.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:10.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:10.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:10.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:10.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:10.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:10.356236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:10.356236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:10.356236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:10.356236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:10.356236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:10.356236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:10.356236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:10.356236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:10.356236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:10.356236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:10.356236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:10.356236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:10.356236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:10.356236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:10.356236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:10.356236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:10.358332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:10.358332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:10.358332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:10.358332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:10.358332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:10.358332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:10.358332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:10.358332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:10.358332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:10.358332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:10.358332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:11:10.358332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:10.358332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 389/793] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:11:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:10.360799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 405/794] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:10.365000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be959eb1-a3f2-4823-8b1b-fc181ce6403f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:10.365142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b1f7f-f93b-4a53-9742-0b9d89fa05b0 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:10.374695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:10.458244 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:10.471446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b1f7f-f93b-4a53-9742-0b9d89fa05b0 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:10.475836 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be959eb1-a3f2-4823-8b1b-fc181ce6403f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:10.507384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d41dfb-29cc-4f96-bca9-4d40346fa7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:10.508418 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d41dfb-29cc-4f96-bca9-4d40346fa7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:10.508909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d41dfb-29cc-4f96-bca9-4d40346fa7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:10.509402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d41dfb-29cc-4f96-bca9-4d40346fa7b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:10.510358 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d41dfb-29cc-4f96-bca9-4d40346fa7b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:10.510865 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d41dfb-29cc-4f96-bca9-4d40346fa7b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:10.512799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d41dfb-29cc-4f96-bca9-4d40346fa7b8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:10.514661 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3da0e-5072-4237-8ed3-e898e38d140f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:10.515388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3da0e-5072-4237-8ed3-e898e38d140f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:10.516145 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3da0e-5072-4237-8ed3-e898e38d140f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:10.516255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3da0e-5072-4237-8ed3-e898e38d140f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:10.523796 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 406/795] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:11:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:10.524936 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d41dfb-29cc-4f96-bca9-4d40346fa7b8 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:10.532986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3da0e-5072-4237-8ed3-e898e38d140f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:10.538833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-6ca3da0e-5072-4237-8ed3-e898e38d140f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:11:10.538833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:10.538833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:10.538833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:10.538833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:10.538833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:10.538833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:10.538833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:10.538833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:10.538833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:10.538833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:10.538833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:10.538833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:10.538833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:10.538833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:10.538833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:10.539779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:10.539779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:10.539779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:10.539779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:10.539779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:10.539779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:10.539779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:10.539779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:10.539779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:10.539779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:10.539779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:10.539779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:10.539779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:10.539779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:10.539779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:10.539779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:10.540620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:10.540620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:10.540620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:10.540620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:10.540620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:10.540620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:10.540620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:10.540620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:10.540620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:10.540620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:10.540620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:11:10.540620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:10.545361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 390/796] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:11:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:10.546370 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3da0e-5072-4237-8ed3-e898e38d140f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:10.597215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:10.637005 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d41dfb-29cc-4f96-bca9-4d40346fa7b8 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:10.654999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3da0e-5072-4237-8ed3-e898e38d140f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:10.675621 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6331c-f7f6-4175-b988-1d18cbddee8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:10.683776 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6331c-f7f6-4175-b988-1d18cbddee8b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:10.683776 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6331c-f7f6-4175-b988-1d18cbddee8b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:10.683776 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6331c-f7f6-4175-b988-1d18cbddee8b None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:10.683776 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6331c-f7f6-4175-b988-1d18cbddee8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:10.683776 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6331c-f7f6-4175-b988-1d18cbddee8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:10.685221 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6331c-f7f6-4175-b988-1d18cbddee8b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:10.695751 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ecf866-28bf-4a06-8ef3-6601930097bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:10.696408 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecf866-28bf-4a06-8ef3-6601930097bf None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:10.696690 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecf866-28bf-4a06-8ef3-6601930097bf None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:10.696961 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ecf866-28bf-4a06-8ef3-6601930097bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:10.703146 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 407/797] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:10.704260 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6331c-f7f6-4175-b988-1d18cbddee8b None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:10.708510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ecf866-28bf-4a06-8ef3-6601930097bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:10.710092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ecf866-28bf-4a06-8ef3-6601930097bf None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:10.717335 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:10.727151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-22ecf866-28bf-4a06-8ef3-6601930097bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:10.727151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:10.727151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:10.727151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:10.727151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:10.727151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:10.727151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:10.727151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:10.727151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:10.727151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:10.727151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:10.727151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:10.727151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:10.727151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:10.727151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:10.728135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:10.729110 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:10.729110 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:10.729110 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:10.729110 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:10.733462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 391/798] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:10.734847 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33922086-9509-4d32-946c-375fb9dfa16e None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:10.735135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33922086-9509-4d32-946c-375fb9dfa16e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:10.735413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33922086-9509-4d32-946c-375fb9dfa16e None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:10.736912 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 392/799] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:10.737513 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33922086-9509-4d32-946c-375fb9dfa16e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:10.819382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6331c-f7f6-4175-b988-1d18cbddee8b None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:10.844410 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33922086-9509-4d32-946c-375fb9dfa16e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:10.846502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:10.856485 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859c4f6-0aa2-4201-8218-058fa0ebb447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:10.857414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859c4f6-0aa2-4201-8218-058fa0ebb447 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:10.857715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859c4f6-0aa2-4201-8218-058fa0ebb447 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:10.858004 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859c4f6-0aa2-4201-8218-058fa0ebb447 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:10.858833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859c4f6-0aa2-4201-8218-058fa0ebb447 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:10.859033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859c4f6-0aa2-4201-8218-058fa0ebb447 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:10.860416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859c4f6-0aa2-4201-8218-058fa0ebb447 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:10.871754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 408/800] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:11:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:10.872585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859c4f6-0aa2-4201-8218-058fa0ebb447 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:10.889551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2bbf8-f2bc-4323-ba8c-cddc1afef760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:10.890246 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2bbf8-f2bc-4323-ba8c-cddc1afef760 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:10.890566 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2bbf8-f2bc-4323-ba8c-cddc1afef760 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:10.890838 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2bbf8-f2bc-4323-ba8c-cddc1afef760 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:10.899608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2bbf8-f2bc-4323-ba8c-cddc1afef760 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:10.901095 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2bbf8-f2bc-4323-ba8c-cddc1afef760 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:10.913830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-7ac2bbf8-f2bc-4323-ba8c-cddc1afef760 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:10.913830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:10.913830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:10.913830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:10.913830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:10.913830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:10.913830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:10.913830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:10.913830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:10.913830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:10.913830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:10.913830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:10.913830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:11:10.913830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:11:10.913830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:10.914891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:10.915797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:11:10.915797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:11:10.915797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:11:10.915797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:11:10.915797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:11:10.915797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:11:10.915797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:10.915797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:10.915797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 393/801] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:10.916419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53a2d3-9336-433d-b6de-46bc33c339db None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:10.916541 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53a2d3-9336-433d-b6de-46bc33c339db None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:10.918114 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53a2d3-9336-433d-b6de-46bc33c339db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:10.936986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-1d53a2d3-9336-433d-b6de-46bc33c339db None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:10.938342 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1d53a2d3-9336-433d-b6de-46bc33c339db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:10.987736 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859c4f6-0aa2-4201-8218-058fa0ebb447 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:10.993607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:11.012950 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 394/802] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:11.012950 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53a2d3-9336-433d-b6de-46bc33c339db None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:11.037590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de5811-c794-4b02-a0c2-023f16e78ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:11.038885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de5811-c794-4b02-a0c2-023f16e78ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:11.039163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de5811-c794-4b02-a0c2-023f16e78ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:11.039433 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de5811-c794-4b02-a0c2-023f16e78ad2 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:11.040135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de5811-c794-4b02-a0c2-023f16e78ad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:11.040434 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de5811-c794-4b02-a0c2-023f16e78ad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:11.041825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de5811-c794-4b02-a0c2-023f16e78ad2 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:11.055097 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 409/803] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:11.056018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de5811-c794-4b02-a0c2-023f16e78ad2 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:11.110375 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:11.137073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53a2d3-9336-433d-b6de-46bc33c339db None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:11.160271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de5811-c794-4b02-a0c2-023f16e78ad2 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:11.175917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6924a2-e5f6-4f97-b9fa-3141bdf7d397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:11.176687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6924a2-e5f6-4f97-b9fa-3141bdf7d397 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:11.176931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6924a2-e5f6-4f97-b9fa-3141bdf7d397 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:11.177233 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6924a2-e5f6-4f97-b9fa-3141bdf7d397 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:11.188535 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6924a2-e5f6-4f97-b9fa-3141bdf7d397 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:11.190189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6924a2-e5f6-4f97-b9fa-3141bdf7d397 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:11.197988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39277bf-329b-4a73-b8c8-435982f82872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:11.198792 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39277bf-329b-4a73-b8c8-435982f82872 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:11.199071 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39277bf-329b-4a73-b8c8-435982f82872 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:11.199339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39277bf-329b-4a73-b8c8-435982f82872 None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:11.200099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39277bf-329b-4a73-b8c8-435982f82872 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:11.200403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39277bf-329b-4a73-b8c8-435982f82872 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:11.201863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39277bf-329b-4a73-b8c8-435982f82872 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:11.207538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-4f6924a2-e5f6-4f97-b9fa-3141bdf7d397 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:11.207538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:11.207538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:11.207538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:11.207538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:11.207538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:11.207538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:11.207538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:11.207538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:11.207538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:11.207538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:11.207538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:11.207538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:11.207538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:11.207538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:11.208419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:11.209319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:11.209319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:11.209319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:11.209319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:11.210829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 395/804] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:11.211825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6924a2-e5f6-4f97-b9fa-3141bdf7d397 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:11.217957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 410/805] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:11:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:11.218823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39277bf-329b-4a73-b8c8-435982f82872 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:11.257359 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:11.316463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6924a2-e5f6-4f97-b9fa-3141bdf7d397 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:11.319229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39277bf-329b-4a73-b8c8-435982f82872 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:11.329674 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:11.350058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4faf698-2bf3-4100-aba5-e18c7c3b967b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:11.350708 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4faf698-2bf3-4100-aba5-e18c7c3b967b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:11.350990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4faf698-2bf3-4100-aba5-e18c7c3b967b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:11.351262 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4faf698-2bf3-4100-aba5-e18c7c3b967b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:11.351878 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4faf698-2bf3-4100-aba5-e18c7c3b967b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:11.353304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4faf698-2bf3-4100-aba5-e18c7c3b967b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:11.353705 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcca9c0-7684-4a51-9424-843b4e75d827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:11.354381 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcca9c0-7684-4a51-9424-843b4e75d827 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:11.354657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcca9c0-7684-4a51-9424-843b4e75d827 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:11.354929 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcca9c0-7684-4a51-9424-843b4e75d827 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:11.362846 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcca9c0-7684-4a51-9424-843b4e75d827 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:11.363072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-c4faf698-2bf3-4100-aba5-e18c7c3b967b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:11:11.363072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:11.363072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:11.363072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:11.363072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:11.363072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:11.363072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:11.363072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:11.363072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:11:11.363072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:11.363072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:11.363072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:11.363072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:11.363072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:11.363072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:11.363072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:11.364044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:11.364044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:11.364044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:11.364044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:11.364044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:11.364044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:11.364044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:11.364044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:11.364044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:11.364044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:11.364044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:11.364044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:11.364044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:11.364044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:11.364044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:11.364044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:11.364876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:11.364876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:11.364876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:11.364876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:11.364876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:11.364876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:11.364876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:11.364876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:11.364876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:11.364876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:11.364876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:11.364876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:11.364876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:11.364876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:11:11.364876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:11.365707 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcca9c0-7684-4a51-9424-843b4e75d827 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:11.365808 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 396/806] 10.222.0.48 () {56 vars in 1129 bytes} [Mon Aug 1 15:11:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:11.365808 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26153cde-78cd-410f-970d-1932f985cd73 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:11.366597 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26153cde-78cd-410f-970d-1932f985cd73 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:11.366597 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26153cde-78cd-410f-970d-1932f985cd73 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:11.367829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 397/807] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:11.368409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26153cde-78cd-410f-970d-1932f985cd73 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:11.382172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-0bcca9c0-7684-4a51-9424-843b4e75d827 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:11:11.382172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:11.382172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:11.382172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:11.382172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:11.382172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:11.382172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:11.382172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:11.382172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:11.382172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:11.382172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:11.382172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:11.382172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:11:11.382172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:11:11.382172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:11:11.382172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:11.383240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:11.384248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:11.384248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:11.384248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:11:11.384248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:11:11.384248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:11:11.384248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:11:11.384248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:11:11.384248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:11:11.384248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:11:11.384248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:11.384248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 411/808] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:11:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:11.384909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcca9c0-7684-4a51-9424-843b4e75d827 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:11.472901 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26153cde-78cd-410f-970d-1932f985cd73 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:11.475984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:11.486808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcca9c0-7684-4a51-9424-843b4e75d827 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:11.509137 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a961053-7b95-455a-9230-5d2d7003601c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:11.509805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a961053-7b95-455a-9230-5d2d7003601c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:11.510077 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a961053-7b95-455a-9230-5d2d7003601c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:11.510344 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a961053-7b95-455a-9230-5d2d7003601c None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:11.511040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a961053-7b95-455a-9230-5d2d7003601c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:11.511340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a961053-7b95-455a-9230-5d2d7003601c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:11.512751 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a961053-7b95-455a-9230-5d2d7003601c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:11.520831 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3b098-7aaf-4db5-ba04-6f2a7a37dd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:11.521510 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3b098-7aaf-4db5-ba04-6f2a7a37dd81 None admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:11.521849 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3b098-7aaf-4db5-ba04-6f2a7a37dd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:11.522069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3b098-7aaf-4db5-ba04-6f2a7a37dd81 None admin] PATH_INFO: `/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/38b18e5d1e8843c6a08b1f5609cc60d5/roles/624a5f7ed1cf440cb177291e89bb11ff` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:11.524406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 398/809] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:11.525186 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a961053-7b95-455a-9230-5d2d7003601c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:11.540799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3b098-7aaf-4db5-ba04-6f2a7a37dd81 None admin] RBAC: Authorizing `identity:create_grant(project_id=218a9ee5cb2842248b6995d2562bb74b, user_id=38b18e5d1e8843c6a08b1f5609cc60d5, role_id=624a5f7ed1cf440cb177291e89bb11ff)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:11.542309 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3b098-7aaf-4db5-ba04-6f2a7a37dd81 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:11.555319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 412/810] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:11:11 2022] PUT /identity/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/38b18e5d1e8843c6a08b1f5609cc60d5/roles/624a5f7ed1cf440cb177291e89bb11ff => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:11:11.556372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdc88d-3e7a-4f22-bd74-1b33367420c2 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:11.556653 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdc88d-3e7a-4f22-bd74-1b33367420c2 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:11.556928 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdc88d-3e7a-4f22-bd74-1b33367420c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:11.578083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-c9fdc88d-3e7a-4f22-bd74-1b33367420c2 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:11.579472 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c9fdc88d-3e7a-4f22-bd74-1b33367420c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:11.599096 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:11.651429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a961053-7b95-455a-9230-5d2d7003601c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:11.660539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 413/811] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:11.661493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdc88d-3e7a-4f22-bd74-1b33367420c2 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:11.701163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fdccd-0cc8-49f2-b360-d886c2db2642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:11.701941 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fdccd-0cc8-49f2-b360-d886c2db2642 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:11.702177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fdccd-0cc8-49f2-b360-d886c2db2642 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:11.702436 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fdccd-0cc8-49f2-b360-d886c2db2642 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:11.703198 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fdccd-0cc8-49f2-b360-d886c2db2642 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:11.703504 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fdccd-0cc8-49f2-b360-d886c2db2642 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:11.704905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fdccd-0cc8-49f2-b360-d886c2db2642 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:11.714151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 399/812] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 15:11:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:11.715215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fdccd-0cc8-49f2-b360-d886c2db2642 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:11.748298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:11.776125 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdc88d-3e7a-4f22-bd74-1b33367420c2 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:11.812631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a075e2-cea3-4b9f-b1dc-a41bb99abf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:11.813641 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a075e2-cea3-4b9f-b1dc-a41bb99abf68 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:11.813955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a075e2-cea3-4b9f-b1dc-a41bb99abf68 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:11.814237 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a075e2-cea3-4b9f-b1dc-a41bb99abf68 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:11.815041 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a075e2-cea3-4b9f-b1dc-a41bb99abf68 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:11.815348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a075e2-cea3-4b9f-b1dc-a41bb99abf68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:11.816786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a075e2-cea3-4b9f-b1dc-a41bb99abf68 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:11.831816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fdccd-0cc8-49f2-b360-d886c2db2642 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:11.839657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 414/813] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 15:11:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:11.839657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a075e2-cea3-4b9f-b1dc-a41bb99abf68 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:11.850179 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:11.870416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd36b5-b8b5-425a-a36a-9aefa1790177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:11.871178 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd36b5-b8b5-425a-a36a-9aefa1790177 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:11.871430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd36b5-b8b5-425a-a36a-9aefa1790177 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:11.871620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd36b5-b8b5-425a-a36a-9aefa1790177 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:11.880409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd36b5-b8b5-425a-a36a-9aefa1790177 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:11.881928 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd36b5-b8b5-425a-a36a-9aefa1790177 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:11.894789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-6bcd36b5-b8b5-425a-a36a-9aefa1790177 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:11:11.894789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:11.894789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:11.894789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:11.894789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:11.894789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:11.894789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:11.894789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:11.894789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:11.894789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:11.894789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:11.894789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:11.894789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:11:11.894789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:11:11.894789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:11:11.894789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:11.895799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:11.896747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:11.896747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:11.896747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:11:11.896747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:11:11.896747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:11:11.896747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:11:11.896747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:11:11.896747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:11:11.896747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:11:11.896747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:11.896747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 400/814] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 15:11:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:11.896747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd36b5-b8b5-425a-a36a-9aefa1790177 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:11.966918 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a075e2-cea3-4b9f-b1dc-a41bb99abf68 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:12.001417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:12.003776 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63535a5b-f26b-41af-bf56-99fbb49cb552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:12.004420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63535a5b-f26b-41af-bf56-99fbb49cb552 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:12.004696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63535a5b-f26b-41af-bf56-99fbb49cb552 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:12.004959 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63535a5b-f26b-41af-bf56-99fbb49cb552 None admin] PATH_INFO: `/v3/services/network` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:12.005592 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63535a5b-f26b-41af-bf56-99fbb49cb552 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:12.006999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63535a5b-f26b-41af-bf56-99fbb49cb552 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-63535a5b-f26b-41af-bf56-99fbb49cb552 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:12.016320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:12.017397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:12.017397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:12.017397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:12.017397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:12.017397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:12.017397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:12.017397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:12.017397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:12.017397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:12.017397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:12.017397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:12.017397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:12.017397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:12.017397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:12.017397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:12.017397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:12.018586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:12.018586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:12.018586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:12.018586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:12.018586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:12.018586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:12.018586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:12.018586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:12.018586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:12.018586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:12.018586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:12.018586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:12.018586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:11:12.018586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:12.018586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 415/815] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:11 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:12.019386 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63535a5b-f26b-41af-bf56-99fbb49cb552 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:12.024984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd36b5-b8b5-425a-a36a-9aefa1790177 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:12.061491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065ed05-1d5c-41a4-a913-4ea28d2d52d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:12.062534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065ed05-1d5c-41a4-a913-4ea28d2d52d9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:12.063418 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065ed05-1d5c-41a4-a913-4ea28d2d52d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:12.064253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065ed05-1d5c-41a4-a913-4ea28d2d52d9 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:12.065461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065ed05-1d5c-41a4-a913-4ea28d2d52d9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:12.066045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065ed05-1d5c-41a4-a913-4ea28d2d52d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:12.067637 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065ed05-1d5c-41a4-a913-4ea28d2d52d9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:12.078605 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 401/816] 10.222.0.48 () {56 vars in 1124 bytes} [Mon Aug 1 15:11:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:12.078605 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065ed05-1d5c-41a4-a913-4ea28d2d52d9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:12.098732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:12.131191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63535a5b-f26b-41af-bf56-99fbb49cb552 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:12.165609 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58c720-d17d-4c5e-96dc-afe18dd71e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:12.166299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58c720-d17d-4c5e-96dc-afe18dd71e58 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:12.166743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58c720-d17d-4c5e-96dc-afe18dd71e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:12.167142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58c720-d17d-4c5e-96dc-afe18dd71e58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:12.178085 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58c720-d17d-4c5e-96dc-afe18dd71e58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:12.179554 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58c720-d17d-4c5e-96dc-afe18dd71e58 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:12.180393 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065ed05-1d5c-41a4-a913-4ea28d2d52d9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:12.193821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-9d58c720-d17d-4c5e-96dc-afe18dd71e58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:12.193821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:12.193821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:12.193821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:12.193821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:12.193821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:12.193821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:12.193821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:12.193821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:12.193821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:12.193821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:12.193821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:12.193821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:12.193821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:12.193821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:12.194766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:12.195737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:12.195737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:12.195737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:12.195737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:12.195737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 416/817] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:12.195737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58c720-d17d-4c5e-96dc-afe18dd71e58 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:12.228202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffaf767-0b55-4aa1-a0d2-7c699dc38051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:12.229491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaf767-0b55-4aa1-a0d2-7c699dc38051 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:12.229981 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaf767-0b55-4aa1-a0d2-7c699dc38051 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:12.230546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaf767-0b55-4aa1-a0d2-7c699dc38051 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:12.231731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffaf767-0b55-4aa1-a0d2-7c699dc38051 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:12.232647 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffaf767-0b55-4aa1-a0d2-7c699dc38051 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:12.235007 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffaf767-0b55-4aa1-a0d2-7c699dc38051 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:12.257586 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 402/818] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 15:11:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:12.258572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffaf767-0b55-4aa1-a0d2-7c699dc38051 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:12.275778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:12.314065 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58c720-d17d-4c5e-96dc-afe18dd71e58 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:12.337808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:12.352037 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1de39a-b3a9-4e7e-a485-1f7fb9fff218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:12.353022 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1de39a-b3a9-4e7e-a485-1f7fb9fff218 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:12.353306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1de39a-b3a9-4e7e-a485-1f7fb9fff218 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:12.353585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1de39a-b3a9-4e7e-a485-1f7fb9fff218 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:12.354631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1de39a-b3a9-4e7e-a485-1f7fb9fff218 None admin] RBAC: Adding query filter params (name=network) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:12.355072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1de39a-b3a9-4e7e-a485-1f7fb9fff218 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:12.356478 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1de39a-b3a9-4e7e-a485-1f7fb9fff218 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:12.361920 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 417/819] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 15:11:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:12.362550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1de39a-b3a9-4e7e-a485-1f7fb9fff218 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:12.377537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffaf767-0b55-4aa1-a0d2-7c699dc38051 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:12.413800 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f96f35-7b37-41d0-9174-618bca8d827c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:12.414426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f96f35-7b37-41d0-9174-618bca8d827c None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:12.414713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f96f35-7b37-41d0-9174-618bca8d827c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:12.414977 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f96f35-7b37-41d0-9174-618bca8d827c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:12.415695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f96f35-7b37-41d0-9174-618bca8d827c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:12.417127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f96f35-7b37-41d0-9174-618bca8d827c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:12.440025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 403/820] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:11:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:11:12.440664 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f96f35-7b37-41d0-9174-618bca8d827c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:12.494639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1de39a-b3a9-4e7e-a485-1f7fb9fff218 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:12.528748 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:12.553905 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d05c1-e9d2-4787-afa5-55fcecf9da67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:12.554878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d05c1-e9d2-4787-afa5-55fcecf9da67 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:12.555375 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d05c1-e9d2-4787-afa5-55fcecf9da67 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:12.555849 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d05c1-e9d2-4787-afa5-55fcecf9da67 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:12.556757 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d05c1-e9d2-4787-afa5-55fcecf9da67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:12.557329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d05c1-e9d2-4787-afa5-55fcecf9da67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:12.559031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d05c1-e9d2-4787-afa5-55fcecf9da67 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:12.576856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f96f35-7b37-41d0-9174-618bca8d827c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:12.577469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 418/821] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:12.578753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d05c1-e9d2-4787-afa5-55fcecf9da67 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:12.633804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70591f58-d6c9-4a1e-ba59-3c3a1583d1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:12.633804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70591f58-d6c9-4a1e-ba59-3c3a1583d1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:12.633804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70591f58-d6c9-4a1e-ba59-3c3a1583d1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:12.633804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70591f58-d6c9-4a1e-ba59-3c3a1583d1a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:12.634510 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:12.702305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d05c1-e9d2-4787-afa5-55fcecf9da67 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:12.720645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70591f58-d6c9-4a1e-ba59-3c3a1583d1a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:12.722382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70591f58-d6c9-4a1e-ba59-3c3a1583d1a3 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:12.743408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b3191-9411-46fc-b27f-ddf1fc46a507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:12.744538 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b3191-9411-46fc-b27f-ddf1fc46a507 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:12.745095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b3191-9411-46fc-b27f-ddf1fc46a507 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:12.745576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b3191-9411-46fc-b27f-ddf1fc46a507 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:12.746488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b3191-9411-46fc-b27f-ddf1fc46a507 None admin] RBAC: Adding query filter params (type=network) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:12.746990 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b3191-9411-46fc-b27f-ddf1fc46a507 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:12.748697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b3191-9411-46fc-b27f-ddf1fc46a507 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:12.759032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 419/822] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 15:11:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:12.760294 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b3191-9411-46fc-b27f-ddf1fc46a507 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:12.803266 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:12.844602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 404/823] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:11:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:11:12.845483 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70591f58-d6c9-4a1e-ba59-3c3a1583d1a3 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:12.873084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b3191-9411-46fc-b27f-ddf1fc46a507 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:12.884282 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:12.917749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc953d-4305-4aff-b125-99080dd4fc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:12.918450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc953d-4305-4aff-b125-99080dd4fc3b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:12.918730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc953d-4305-4aff-b125-99080dd4fc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:12.919003 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc953d-4305-4aff-b125-99080dd4fc3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:12.929131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc953d-4305-4aff-b125-99080dd4fc3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:12.931268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc953d-4305-4aff-b125-99080dd4fc3b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:12.946008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-08fc953d-4305-4aff-b125-99080dd4fc3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:12.946008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:12.946008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:12.946008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:12.946008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:12.946008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:12.946008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:12.946008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:12.946008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:12.946008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:12.946008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:12.946008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:12.946008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:11:12.946008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:11:12.946008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:12.946957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:12.948032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:11:12.948032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:11:12.948032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:11:12.948032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:11:12.948032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:11:12.948032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:11:12.948032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:12.948032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:12.949992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 420/824] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:12.951001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc953d-4305-4aff-b125-99080dd4fc3b None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:12.988660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70591f58-d6c9-4a1e-ba59-3c3a1583d1a3 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:13.026854 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01033a75-07ca-4006-903c-358d6f3771cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:13.027939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01033a75-07ca-4006-903c-358d6f3771cc None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:13.028412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01033a75-07ca-4006-903c-358d6f3771cc None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:13.028875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01033a75-07ca-4006-903c-358d6f3771cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:13.029892 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01033a75-07ca-4006-903c-358d6f3771cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:13.031660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01033a75-07ca-4006-903c-358d6f3771cc None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:13.057917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 405/825] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:11:12 2022] POST /identity/v3/endpoints => generated 327 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:11:13.059316 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01033a75-07ca-4006-903c-358d6f3771cc None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:13.073544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc953d-4305-4aff-b125-99080dd4fc3b None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:13.127630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:13.148890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9026e2-272c-4cb1-a231-61b4bf691c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:13.150034 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9026e2-272c-4cb1-a231-61b4bf691c21 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:13.150615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9026e2-272c-4cb1-a231-61b4bf691c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:13.151128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9026e2-272c-4cb1-a231-61b4bf691c21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:13.165337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9026e2-272c-4cb1-a231-61b4bf691c21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:13.167367 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9026e2-272c-4cb1-a231-61b4bf691c21 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:13.186030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-0e9026e2-272c-4cb1-a231-61b4bf691c21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:13.186030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:13.186030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:13.186030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:13.186030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:13.186030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:13.186030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:13.186030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:13.186030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:13.186030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:13.186030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:13.186030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:13.186030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:13.186030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:13.186030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:13.187014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:13.188062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:13.188062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:13.188062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:13.188062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:13.191717 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 421/826] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:13.197859 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9026e2-272c-4cb1-a231-61b4bf691c21 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:13.236419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:13.308979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01033a75-07ca-4006-903c-358d6f3771cc None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:13.348418 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560a46a-e856-49cc-afd1-5dc43a7c2911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:13.349056 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560a46a-e856-49cc-afd1-5dc43a7c2911 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:13.349344 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560a46a-e856-49cc-afd1-5dc43a7c2911 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:13.349619 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560a46a-e856-49cc-afd1-5dc43a7c2911 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:13.350517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560a46a-e856-49cc-afd1-5dc43a7c2911 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:13.350829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560a46a-e856-49cc-afd1-5dc43a7c2911 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:13.352447 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560a46a-e856-49cc-afd1-5dc43a7c2911 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:13.358393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9026e2-272c-4cb1-a231-61b4bf691c21 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:13.366434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 406/827] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:11:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:13.372884 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560a46a-e856-49cc-afd1-5dc43a7c2911 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:13.399223 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0573af-64be-4b0e-8f3e-b40da69ddc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:13.400372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0573af-64be-4b0e-8f3e-b40da69ddc41 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:13.401218 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0573af-64be-4b0e-8f3e-b40da69ddc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:13.401976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0573af-64be-4b0e-8f3e-b40da69ddc41 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:13.402891 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0573af-64be-4b0e-8f3e-b40da69ddc41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:13.403540 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0573af-64be-4b0e-8f3e-b40da69ddc41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:13.405723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0573af-64be-4b0e-8f3e-b40da69ddc41 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:13.421644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 422/828] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:13.426706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0573af-64be-4b0e-8f3e-b40da69ddc41 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:13.466831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:13.488054 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560a46a-e856-49cc-afd1-5dc43a7c2911 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:13.512240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:13.529712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd473dca-150c-416d-9a05-443a51bb6449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:13.530574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd473dca-150c-416d-9a05-443a51bb6449 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:13.530997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd473dca-150c-416d-9a05-443a51bb6449 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:13.531375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd473dca-150c-416d-9a05-443a51bb6449 None admin] PATH_INFO: `/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/25a26f54accf4264aef646c3757de7e9/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:13.533499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd473dca-150c-416d-9a05-443a51bb6449 None admin] RBAC: Authorizing `identity:create_grant(project_id=218a9ee5cb2842248b6995d2562bb74b, user_id=25a26f54accf4264aef646c3757de7e9, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:13.535071 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd473dca-150c-416d-9a05-443a51bb6449 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:13.543811 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0573af-64be-4b0e-8f3e-b40da69ddc41 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:13.551940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 407/829] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:11:13 2022] PUT /identity/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/25a26f54accf4264aef646c3757de7e9/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:11:13.587517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9ddd4-1c7b-416d-af8f-2e0d3aef62e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:13.588114 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9ddd4-1c7b-416d-af8f-2e0d3aef62e8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:13.588396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9ddd4-1c7b-416d-af8f-2e0d3aef62e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:13.588669 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9ddd4-1c7b-416d-af8f-2e0d3aef62e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:13.597546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2f99f-1226-4d2c-b7b3-fc4293abf453 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:13.597895 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2f99f-1226-4d2c-b7b3-fc4293abf453 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:13.598275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2f99f-1226-4d2c-b7b3-fc4293abf453 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:13.599676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9ddd4-1c7b-416d-af8f-2e0d3aef62e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:13.600208 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 408/830] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:11:13.601307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9ddd4-1c7b-416d-af8f-2e0d3aef62e8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:13.606085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfc88d-b091-4798-b6a4-6516b21ba012 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:13.606503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfc88d-b091-4798-b6a4-6516b21ba012 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:13.606843 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfc88d-b091-4798-b6a4-6516b21ba012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:13.619450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-c3c9ddd4-1c7b-416d-af8f-2e0d3aef62e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:13.619450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:13.619450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:13.619450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:13.619450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:13.619450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:13.619450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:13.619450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:13.619450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:13.619450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:13.619450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:13.619450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:13.619450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:11:13.619450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:11:13.619450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:13.620607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:13.621860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:11:13.621860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:11:13.621860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:11:13.621860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:11:13.621860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:11:13.621860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:11:13.621860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:13.621860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:13.621860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 423/831] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:13.628435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9ddd4-1c7b-416d-af8f-2e0d3aef62e8 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:13.629197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-45cfc88d-b091-4798-b6a4-6516b21ba012 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:13.630545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-45cfc88d-b091-4798-b6a4-6516b21ba012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:13.726067 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 409/832] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:11:13.754507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfc88d-b091-4798-b6a4-6516b21ba012 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:13.777991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9ddd4-1c7b-416d-af8f-2e0d3aef62e8 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:13.824355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956ca83-401e-4240-a049-218279860c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:13.825140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956ca83-401e-4240-a049-218279860c03 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:13.825544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956ca83-401e-4240-a049-218279860c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:13.825861 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956ca83-401e-4240-a049-218279860c03 None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:13.826889 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956ca83-401e-4240-a049-218279860c03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:13.827628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956ca83-401e-4240-a049-218279860c03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:13.829390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956ca83-401e-4240-a049-218279860c03 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:13.833707 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:13.843384 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:13.849390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 424/833] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:11:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:13.856314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956ca83-401e-4240-a049-218279860c03 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:13.885754 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfc88d-b091-4798-b6a4-6516b21ba012 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:13.923559 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27169b98-1b3d-40d2-88e0-59311ba9a808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:13.924538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27169b98-1b3d-40d2-88e0-59311ba9a808 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:13.924961 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27169b98-1b3d-40d2-88e0-59311ba9a808 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:13.925367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27169b98-1b3d-40d2-88e0-59311ba9a808 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:13.936432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27169b98-1b3d-40d2-88e0-59311ba9a808 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:13.941749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-27169b98-1b3d-40d2-88e0-59311ba9a808 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:11:13.941749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:13.941749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:13.941749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:13.941749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:13.941749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:13.941749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:13.941749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:13.941749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:13.941749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:13.941749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:13.941749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:13.941749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:13.941749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:13.941749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:13.941749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:13.942917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:13.942917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:13.942917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:13.942917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:13.942917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:13.942917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:13.942917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:13.942917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:13.942917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:13.942917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:13.942917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:13.942917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:13.942917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:13.942917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:13.942917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:13.942917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:13.943967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:13.943967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:13.943967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:13.943967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:13.943967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:13.943967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:13.943967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:13.943967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:13.943967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:13.943967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:13.943967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:11:13.943967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:13.945863 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 410/834] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:11:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:13.951752 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27169b98-1b3d-40d2-88e0-59311ba9a808 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:13.972935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956ca83-401e-4240-a049-218279860c03 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:14.020295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30a0bb-b7f6-4910-be9d-5763a385e2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:14.021306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30a0bb-b7f6-4910-be9d-5763a385e2eb None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:14.021722 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30a0bb-b7f6-4910-be9d-5763a385e2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:14.022166 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f30a0bb-b7f6-4910-be9d-5763a385e2eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:14.023364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30a0bb-b7f6-4910-be9d-5763a385e2eb None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=2de0ff99a2b74979a28a8b10e9ebdfcd) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:14.023848 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30a0bb-b7f6-4910-be9d-5763a385e2eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:14.026449 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f30a0bb-b7f6-4910-be9d-5763a385e2eb None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:14.037642 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 425/835] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:11:13 2022] GET /identity/v3/role_assignments?user.id=2de0ff99a2b74979a28a8b10e9ebdfcd&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:14.068247 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27169b98-1b3d-40d2-88e0-59311ba9a808 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:14.110042 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:14.116445 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc3952-feb4-4cf3-9445-1705ca4adb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:14.117377 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc3952-feb4-4cf3-9445-1705ca4adb97 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:14.117830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc3952-feb4-4cf3-9445-1705ca4adb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:14.118408 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc3952-feb4-4cf3-9445-1705ca4adb97 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:14.119286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc3952-feb4-4cf3-9445-1705ca4adb97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:14.119725 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc3952-feb4-4cf3-9445-1705ca4adb97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:14.121316 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc3952-feb4-4cf3-9445-1705ca4adb97 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:14.131649 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 411/836] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:11:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:14.139002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30a0bb-b7f6-4910-be9d-5763a385e2eb None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:14.207941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:14.254036 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f30a0bb-b7f6-4910-be9d-5763a385e2eb None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:14.295329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b67ff9-d00e-448c-b23d-1ba81f9b4980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:14.295993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b67ff9-d00e-448c-b23d-1ba81f9b4980 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:14.296275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b67ff9-d00e-448c-b23d-1ba81f9b4980 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:14.296651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b67ff9-d00e-448c-b23d-1ba81f9b4980 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:14.309135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b67ff9-d00e-448c-b23d-1ba81f9b4980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:14.310641 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b67ff9-d00e-448c-b23d-1ba81f9b4980 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:14.326185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-84b67ff9-d00e-448c-b23d-1ba81f9b4980 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:14.326185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:14.326185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:14.326185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:14.326185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:14.326185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:14.326185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:14.326185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:14.326185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:14.326185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:14.326185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:14.326185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:14.326185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:14.326185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:14.326185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:14.327110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:14.328620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:14.328620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:14.328620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:14.328620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:14.328620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 426/837] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:14.333057 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc3952-feb4-4cf3-9445-1705ca4adb97 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:14.437692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc3952-feb4-4cf3-9445-1705ca4adb97 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:14.472209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf09f5e-53a7-401c-ae08-a572dc3fd961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:14.472841 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf09f5e-53a7-401c-ae08-a572dc3fd961 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:14.473120 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf09f5e-53a7-401c-ae08-a572dc3fd961 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:14.473394 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf09f5e-53a7-401c-ae08-a572dc3fd961 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:14.474134 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf09f5e-53a7-401c-ae08-a572dc3fd961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:14.474438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf09f5e-53a7-401c-ae08-a572dc3fd961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:14.475833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf09f5e-53a7-401c-ae08-a572dc3fd961 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:14.489125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 412/838] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:14.494546 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b67ff9-d00e-448c-b23d-1ba81f9b4980 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:14.573824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995e45f-5569-4d0f-bfbc-8501a6a1c00a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:14.574091 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995e45f-5569-4d0f-bfbc-8501a6a1c00a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:14.574373 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995e45f-5569-4d0f-bfbc-8501a6a1c00a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:14.576005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 413/839] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:11:14.581094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854084d-9240-418b-9548-4cd02ec721c4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:14.581378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854084d-9240-418b-9548-4cd02ec721c4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:14.581653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854084d-9240-418b-9548-4cd02ec721c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:14.599933 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2854084d-9240-418b-9548-4cd02ec721c4 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:14.601145 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2854084d-9240-418b-9548-4cd02ec721c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:14.603898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b67ff9-d00e-448c-b23d-1ba81f9b4980 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:14.621529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:14.649587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d666a47-1a63-4465-9e81-e57958abcf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:14.650634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d666a47-1a63-4465-9e81-e57958abcf62 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:14.651133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d666a47-1a63-4465-9e81-e57958abcf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:14.651414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d666a47-1a63-4465-9e81-e57958abcf62 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:14.656297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:14.661751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d666a47-1a63-4465-9e81-e57958abcf62 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:14.663530 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d666a47-1a63-4465-9e81-e57958abcf62 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:14.681955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-6d666a47-1a63-4465-9e81-e57958abcf62 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:11:14.681955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:14.681955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:14.681955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:14.681955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:14.681955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:14.681955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:14.681955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:14.681955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:14.681955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:14.681955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:14.681955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:14.681955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:11:14.681955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:11:14.681955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:11:14.681955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:14.682957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:14.683898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:14.683898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:14.683898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:11:14.683898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:11:14.683898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:11:14.683898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:11:14.683898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:11:14.683898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:11:14.683898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:11:14.683898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:14.683898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 427/841] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 15:11:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:14.684512 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 414/841] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:11:14.687284 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d666a47-1a63-4465-9e81-e57958abcf62 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:14.703126 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854084d-9240-418b-9548-4cd02ec721c4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:14.801695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d666a47-1a63-4465-9e81-e57958abcf62 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:14.841393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca35e2-dcd9-4dec-924b-9c4ed5f5b5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:14.842505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca35e2-dcd9-4dec-924b-9c4ed5f5b5ed None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:14.842929 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca35e2-dcd9-4dec-924b-9c4ed5f5b5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:14.843375 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca35e2-dcd9-4dec-924b-9c4ed5f5b5ed None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:14.844367 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca35e2-dcd9-4dec-924b-9c4ed5f5b5ed None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:14.844838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca35e2-dcd9-4dec-924b-9c4ed5f5b5ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:14.845235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854084d-9240-418b-9548-4cd02ec721c4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:14.846755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca35e2-dcd9-4dec-924b-9c4ed5f5b5ed None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:14.873939 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 428/842] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 15:11:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:14.881012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca35e2-dcd9-4dec-924b-9c4ed5f5b5ed None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:14.893040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc081c37-d72a-4e54-be42-45d74c08bc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:14.893963 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc081c37-d72a-4e54-be42-45d74c08bc2a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:14.894390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc081c37-d72a-4e54-be42-45d74c08bc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:14.894805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc081c37-d72a-4e54-be42-45d74c08bc2a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:14.895580 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc081c37-d72a-4e54-be42-45d74c08bc2a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:14.897145 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc081c37-d72a-4e54-be42-45d74c08bc2a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-bc081c37-d72a-4e54-be42-45d74c08bc2a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:14.906683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:14.907903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:14.907903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:14.907903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:14.907903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:14.907903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:14.907903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:14.907903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:14.907903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:14.907903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:14.907903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:14.907903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:14.907903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:14.907903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:14.907903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:14.907903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:14.907903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:14.908781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:14.908781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:14.908781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:14.908781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:14.908781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:14.908781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:14.908781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:14.908781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:14.908781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:14.908781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:14.908781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:14.908781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:14.908781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:11:14.908781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:14.915842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 415/843] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:11:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:14.916299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:14.917483 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc081c37-d72a-4e54-be42-45d74c08bc2a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:14.969905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:15.006113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca35e2-dcd9-4dec-924b-9c4ed5f5b5ed None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:15.024328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc081c37-d72a-4e54-be42-45d74c08bc2a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:15.048857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca195036-807b-4173-9a41-4014916127ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:15.049760 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca195036-807b-4173-9a41-4014916127ff None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:15.050079 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca195036-807b-4173-9a41-4014916127ff None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:15.050356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca195036-807b-4173-9a41-4014916127ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:15.060071 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7b31a-398d-43be-8bde-efb5ae768931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:15.060990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7b31a-398d-43be-8bde-efb5ae768931 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:15.061414 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7b31a-398d-43be-8bde-efb5ae768931 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:15.061858 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7b31a-398d-43be-8bde-efb5ae768931 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:15.062720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7b31a-398d-43be-8bde-efb5ae768931 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:15.063182 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7b31a-398d-43be-8bde-efb5ae768931 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:15.064818 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7b31a-398d-43be-8bde-efb5ae768931 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:15.074763 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 416/844] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:11:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:15.076327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7b31a-398d-43be-8bde-efb5ae768931 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:15.134956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca195036-807b-4173-9a41-4014916127ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:15.136718 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca195036-807b-4173-9a41-4014916127ff None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:15.174932 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:15.224966 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7b31a-398d-43be-8bde-efb5ae768931 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:15.241981 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:15.268707 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145c261-8512-45e7-8411-f040766bee6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:15.269341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145c261-8512-45e7-8411-f040766bee6b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:15.269614 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145c261-8512-45e7-8411-f040766bee6b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:15.269921 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145c261-8512-45e7-8411-f040766bee6b None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:15.270606 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145c261-8512-45e7-8411-f040766bee6b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:15.270934 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145c261-8512-45e7-8411-f040766bee6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:15.272297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145c261-8512-45e7-8411-f040766bee6b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:15.278101 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 417/846] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:11:15 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:15.279452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 429/846] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:11:14 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:11:15.280488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145c261-8512-45e7-8411-f040766bee6b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:15.398725 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145c261-8512-45e7-8411-f040766bee6b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:15.435685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba83d3-2796-497b-a447-0f86b59d7e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:15.436350 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba83d3-2796-497b-a447-0f86b59d7e20 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:15.436625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba83d3-2796-497b-a447-0f86b59d7e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:15.436895 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba83d3-2796-497b-a447-0f86b59d7e20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:15.449100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba83d3-2796-497b-a447-0f86b59d7e20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:15.450610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba83d3-2796-497b-a447-0f86b59d7e20 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:15.466253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-54ba83d3-2796-497b-a447-0f86b59d7e20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:15.466253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:15.466253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:15.466253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:15.466253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:15.466253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:15.466253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:15.466253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:15.466253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:15.466253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:15.466253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:15.466253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:15.466253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:15.466253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:15.466253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:15.467209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:15.468133 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:15.468133 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:15.468133 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:15.468133 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:15.468133 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 418/847] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:15.474182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca195036-807b-4173-9a41-4014916127ff None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:15.580177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca195036-807b-4173-9a41-4014916127ff None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:15.611299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:15.617743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6a367-adc5-4469-b094-80f360e38d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:15.618638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6a367-adc5-4469-b094-80f360e38d54 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:15.619076 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6a367-adc5-4469-b094-80f360e38d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:15.619498 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6a367-adc5-4469-b094-80f360e38d54 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:15.620382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6a367-adc5-4469-b094-80f360e38d54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:15.620913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6a367-adc5-4469-b094-80f360e38d54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:15.622483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6a367-adc5-4469-b094-80f360e38d54 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:15.635223 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 430/848] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:15.640731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba83d3-2796-497b-a447-0f86b59d7e20 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:15.671654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:15.693814 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a98d13-9d59-4ab5-8913-895f647b4ab6 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:15.694110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a98d13-9d59-4ab5-8913-895f647b4ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:15.694387 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a98d13-9d59-4ab5-8913-895f647b4ab6 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:15.695885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 431/849] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:15.701354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbb00f-02eb-461b-b0cb-11464180d84f None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:15.701859 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbb00f-02eb-461b-b0cb-11464180d84f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:15.702388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbb00f-02eb-461b-b0cb-11464180d84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:15.721725 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-9fbbb00f-02eb-461b-b0cb-11464180d84f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:15.723282 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9fbbb00f-02eb-461b-b0cb-11464180d84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:15.750091 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba83d3-2796-497b-a447-0f86b59d7e20 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:15.787868 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613d99c-95ba-4c5d-8a71-0887e325b4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:15.788628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613d99c-95ba-4c5d-8a71-0887e325b4df None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:15.788917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613d99c-95ba-4c5d-8a71-0887e325b4df None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:15.789197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613d99c-95ba-4c5d-8a71-0887e325b4df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:15.795993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 432/850] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:11:15.797433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613d99c-95ba-4c5d-8a71-0887e325b4df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:15.798955 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613d99c-95ba-4c5d-8a71-0887e325b4df None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:15.818293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-9613d99c-95ba-4c5d-8a71-0887e325b4df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:15.818293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:15.818293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:15.818293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:15.818293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:15.818293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:15.818293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:15.818293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:15.818293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:15.818293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:15.818293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:15.818293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:15.818293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:11:15.818293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:11:15.818293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:15.819432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:15.820336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:11:15.820336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:11:15.820336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:11:15.820336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:11:15.820336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:11:15.820336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:11:15.820336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:15.820336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:15.822187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbb00f-02eb-461b-b0cb-11464180d84f None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:15.822810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 419/851] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:15.828578 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613d99c-95ba-4c5d-8a71-0887e325b4df None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:15.942802 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbb00f-02eb-461b-b0cb-11464180d84f None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:15.943427 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:15.955389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613d99c-95ba-4c5d-8a71-0887e325b4df None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:15.969461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:15.981657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119193a-698d-4e9c-a778-e323c0d3cddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:15.982820 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119193a-698d-4e9c-a778-e323c0d3cddf None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:15.983168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119193a-698d-4e9c-a778-e323c0d3cddf None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:15.983511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119193a-698d-4e9c-a778-e323c0d3cddf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:15.992540 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f2b04-84ae-4153-8de0-97df06df1fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:15.993179 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f2b04-84ae-4153-8de0-97df06df1fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:15.993468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f2b04-84ae-4153-8de0-97df06df1fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:15.993752 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f2b04-84ae-4153-8de0-97df06df1fc0 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:15.994549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f2b04-84ae-4153-8de0-97df06df1fc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:15.994861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f2b04-84ae-4153-8de0-97df06df1fc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:15.995334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119193a-698d-4e9c-a778-e323c0d3cddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:15.996332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f2b04-84ae-4153-8de0-97df06df1fc0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:16.000629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-5119193a-698d-4e9c-a778-e323c0d3cddf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:11:16.000629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:16.000629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:16.000629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:16.000629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:16.000629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:16.000629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:16.000629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:16.000629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:16.000629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:16.000629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:16.000629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:16.000629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:16.000629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:16.000629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:16.000629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:16.001590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:16.001590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:16.001590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:16.001590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:16.001590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:16.001590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:16.001590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:16.001590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:16.001590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:16.001590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:16.001590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:16.001590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:16.001590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:16.001590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:16.001590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:16.001590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:16.002651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:16.002651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:16.002651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:16.002651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:16.002651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:16.002651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:16.002651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:16.002651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:16.002651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:16.002651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:16.002651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:11:16.002651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:16.002651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 433/852] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:11:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:16.009359 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119193a-698d-4e9c-a778-e323c0d3cddf None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:16.015333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 420/853] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:11:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:16.016409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f2b04-84ae-4153-8de0-97df06df1fc0 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:16.122298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119193a-698d-4e9c-a778-e323c0d3cddf None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:16.123070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f2b04-84ae-4153-8de0-97df06df1fc0 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:16.159739 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067122d8-ecc0-49c8-8c87-b029a1c05624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:16.160054 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce07b7f-8907-447a-a79f-9a7a43a2a380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:16.160472 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce07b7f-8907-447a-a79f-9a7a43a2a380 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:16.160639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067122d8-ecc0-49c8-8c87-b029a1c05624 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:16.160810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce07b7f-8907-447a-a79f-9a7a43a2a380 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:16.161014 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce07b7f-8907-447a-a79f-9a7a43a2a380 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:16.161181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067122d8-ecc0-49c8-8c87-b029a1c05624 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:16.162939 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067122d8-ecc0-49c8-8c87-b029a1c05624 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:16.163106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce07b7f-8907-447a-a79f-9a7a43a2a380 None admin] RBAC: Adding query filter params (role.id=624a5f7ed1cf440cb177291e89bb11ff, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=38b18e5d1e8843c6a08b1f5609cc60d5) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:16.163106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce07b7f-8907-447a-a79f-9a7a43a2a380 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:16.163431 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067122d8-ecc0-49c8-8c87-b029a1c05624 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:16.163431 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067122d8-ecc0-49c8-8c87-b029a1c05624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:16.163720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce07b7f-8907-447a-a79f-9a7a43a2a380 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:16.164419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067122d8-ecc0-49c8-8c87-b029a1c05624 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:16.174088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 434/854] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:11:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:16.175342 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 421/855] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:11:16 2022] GET /identity/v3/role_assignments?user.id=38b18e5d1e8843c6a08b1f5609cc60d5&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=624a5f7ed1cf440cb177291e89bb11ff => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:16.183800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067122d8-ecc0-49c8-8c87-b029a1c05624 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:16.229932 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:16.254108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e28351-1a9c-41c9-abf1-671586629ce9 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:16.254624 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e28351-1a9c-41c9-abf1-671586629ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:16.255124 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e28351-1a9c-41c9-abf1-671586629ce9 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:16.261322 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 422/856] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:16.263629 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bbe98-1153-4df5-bed6-a5463e9d2df9 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:16.263952 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bbe98-1153-4df5-bed6-a5463e9d2df9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:16.264395 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bbe98-1153-4df5-bed6-a5463e9d2df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:16.284846 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b61bbe98-1153-4df5-bed6-a5463e9d2df9 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:16.286108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b61bbe98-1153-4df5-bed6-a5463e9d2df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:16.294529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:16.312851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067122d8-ecc0-49c8-8c87-b029a1c05624 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:16.353660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a971a51-3563-4bd1-9ab6-6492d0beb2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:16.354760 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a971a51-3563-4bd1-9ab6-6492d0beb2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:16.355330 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a971a51-3563-4bd1-9ab6-6492d0beb2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:16.356000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a971a51-3563-4bd1-9ab6-6492d0beb2e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:16.362126 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 423/857] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:16.368760 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a971a51-3563-4bd1-9ab6-6492d0beb2e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:16.370484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a971a51-3563-4bd1-9ab6-6492d0beb2e6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:16.386929 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bbe98-1153-4df5-bed6-a5463e9d2df9 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:16.387834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-4a971a51-3563-4bd1-9ab6-6492d0beb2e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:16.387834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:16.387834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:16.387834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:16.387834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:16.387834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:16.387834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:16.387834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:16.387834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:16.387834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:16.387834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:16.387834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:16.387834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:16.387834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:16.387834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:16.388697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:16.389605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:16.389605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:16.389605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:16.389605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:16.391588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 435/858] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:16.397754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a971a51-3563-4bd1-9ab6-6492d0beb2e6 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:16.509695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:16.520936 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a971a51-3563-4bd1-9ab6-6492d0beb2e6 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:16.536150 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bbe98-1153-4df5-bed6-a5463e9d2df9 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:16.559790 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4330b19-5451-4a74-a979-5c334f3a8b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:16.560428 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4330b19-5451-4a74-a979-5c334f3a8b0e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:16.560708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4330b19-5451-4a74-a979-5c334f3a8b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:16.560980 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4330b19-5451-4a74-a979-5c334f3a8b0e None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:16.561686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4330b19-5451-4a74-a979-5c334f3a8b0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:16.562062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4330b19-5451-4a74-a979-5c334f3a8b0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:16.563461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4330b19-5451-4a74-a979-5c334f3a8b0e None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:16.576516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 436/859] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:16.581955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4330b19-5451-4a74-a979-5c334f3a8b0e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:16.584039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe6fcf-0261-4df2-ba98-ab8784941019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:16.584698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe6fcf-0261-4df2-ba98-ab8784941019 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:16.585278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe6fcf-0261-4df2-ba98-ab8784941019 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:16.585556 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe6fcf-0261-4df2-ba98-ab8784941019 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:16.587206 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe6fcf-0261-4df2-ba98-ab8784941019 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:16.588613 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe6fcf-0261-4df2-ba98-ab8784941019 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:16.601089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d5fe6fcf-0261-4df2-ba98-ab8784941019 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:11:16.601089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:16.601089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:16.601089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:16.601089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:16.601089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:16.601089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:16.601089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:16.601089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:11:16.601089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:16.601089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:16.601089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:16.601089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:16.601089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:16.601089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:16.601089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:16.602292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:16.602292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:16.602292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:16.602292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:16.602292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:16.602292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:16.602292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:16.602292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:16.602292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:16.602292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:16.602292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:16.602292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:16.602292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:16.602292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:16.602292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:16.602292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:16.603327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:16.603327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:16.603327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:16.603327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:16.603327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:16.603327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:16.603327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:16.603327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:16.603327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:16.603327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:16.603327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:16.603327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:16.603327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:16.603327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:11:16.603327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:16.603327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 424/860] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Aug 1 15:11:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:16.609568 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe6fcf-0261-4df2-ba98-ab8784941019 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:16.619334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:16.692007 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4330b19-5451-4a74-a979-5c334f3a8b0e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:16.719188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe6fcf-0261-4df2-ba98-ab8784941019 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:16.727905 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b55da-2b85-463f-ba8d-dac40dde1845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:16.728566 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b55da-2b85-463f-ba8d-dac40dde1845 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:16.728846 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b55da-2b85-463f-ba8d-dac40dde1845 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:16.729122 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b55da-2b85-463f-ba8d-dac40dde1845 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:16.737243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784b55da-2b85-463f-ba8d-dac40dde1845 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:16.738738 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784b55da-2b85-463f-ba8d-dac40dde1845 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:16.751220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-784b55da-2b85-463f-ba8d-dac40dde1845 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:11:16.751220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:16.751220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:16.751220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:16.751220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:16.751220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:16.751220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:16.751220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:16.751220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:16.751220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:16.751220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:16.751220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:16.751220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:11:16.751220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:11:16.751220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:11:16.751220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:16.752382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:16.753518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:16.753518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:16.753518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:11:16.753518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:11:16.753518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:11:16.753518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:11:16.753518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:11:16.753518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:11:16.753518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:11:16.753518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:16.753518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 437/861] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:11:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:16.758505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b55da-2b85-463f-ba8d-dac40dde1845 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:16.761084 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2bfad-501c-48f5-b7bb-bf2cb3ad0ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:16.761975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2bfad-501c-48f5-b7bb-bf2cb3ad0ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:16.763935 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2bfad-501c-48f5-b7bb-bf2cb3ad0ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:16.764357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2bfad-501c-48f5-b7bb-bf2cb3ad0ab9 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:16.765236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2bfad-501c-48f5-b7bb-bf2cb3ad0ab9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:16.765674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2bfad-501c-48f5-b7bb-bf2cb3ad0ab9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:16.767240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2bfad-501c-48f5-b7bb-bf2cb3ad0ab9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:16.777154 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:16.777380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 425/862] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 15:11:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:16.778454 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2bfad-501c-48f5-b7bb-bf2cb3ad0ab9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:16.857478 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:16.863792 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b55da-2b85-463f-ba8d-dac40dde1845 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:16.893664 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2bfad-501c-48f5-b7bb-bf2cb3ad0ab9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:16.898759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf5b79-7ee8-4a58-9103-4a0eb3318e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:16.899397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf5b79-7ee8-4a58-9103-4a0eb3318e0c None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:16.899677 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf5b79-7ee8-4a58-9103-4a0eb3318e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:16.899954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf5b79-7ee8-4a58-9103-4a0eb3318e0c None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:16.900774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf5b79-7ee8-4a58-9103-4a0eb3318e0c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:16.901134 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf5b79-7ee8-4a58-9103-4a0eb3318e0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:16.902611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf5b79-7ee8-4a58-9103-4a0eb3318e0c None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:16.925468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 438/863] 10.222.0.48 () {56 vars in 1135 bytes} [Mon Aug 1 15:11:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:16.926415 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf5b79-7ee8-4a58-9103-4a0eb3318e0c None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:16.933758 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f6a72-dcc2-4ffb-92bf-62e85a7a5859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:16.934477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f6a72-dcc2-4ffb-92bf-62e85a7a5859 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:16.934757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f6a72-dcc2-4ffb-92bf-62e85a7a5859 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:16.935025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f6a72-dcc2-4ffb-92bf-62e85a7a5859 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:16.935807 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f6a72-dcc2-4ffb-92bf-62e85a7a5859 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:16.936110 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f6a72-dcc2-4ffb-92bf-62e85a7a5859 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:16.937566 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f6a72-dcc2-4ffb-92bf-62e85a7a5859 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:16.943117 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 426/864] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 15:11:16 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:17.005469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:17.039121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf5b79-7ee8-4a58-9103-4a0eb3318e0c None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:17.077253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdec72-b35c-4b6a-8f15-ce9c44fda123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:17.078000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdec72-b35c-4b6a-8f15-ce9c44fda123 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:17.078283 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdec72-b35c-4b6a-8f15-ce9c44fda123 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:17.078560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdec72-b35c-4b6a-8f15-ce9c44fda123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:17.153810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fdec72-b35c-4b6a-8f15-ce9c44fda123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:17.155355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fdec72-b35c-4b6a-8f15-ce9c44fda123 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:17.229221 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:17.262432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 439/865] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:11:16 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:11:17.268364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f6a72-dcc2-4ffb-92bf-62e85a7a5859 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:17.377807 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f6a72-dcc2-4ffb-92bf-62e85a7a5859 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:17.413547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa012938-04e0-499b-bf45-c71170b307cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:17.414263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012938-04e0-499b-bf45-c71170b307cb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:17.414546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012938-04e0-499b-bf45-c71170b307cb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:17.414817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa012938-04e0-499b-bf45-c71170b307cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:17.418919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:17.430449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa012938-04e0-499b-bf45-c71170b307cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:17.432279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa012938-04e0-499b-bf45-c71170b307cb None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:17.447982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-aa012938-04e0-499b-bf45-c71170b307cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:17.447982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:17.447982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:17.447982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:17.447982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:17.447982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:17.447982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:17.447982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:17.447982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:17.447982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:17.447982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:17.447982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:17.447982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:17.447982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:17.447982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:17.450089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:17.452171 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:17.452171 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:17.452171 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:17.452171 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:17.452725 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 427/866] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:17.455449 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdec72-b35c-4b6a-8f15-ce9c44fda123 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:17.469336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916bcbf-db35-4d97-938a-32f7dce07be6 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:17.469336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916bcbf-db35-4d97-938a-32f7dce07be6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:17.469336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916bcbf-db35-4d97-938a-32f7dce07be6 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:17.470226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 428/867] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:17.475644 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c4376-dbb8-4c38-b2f9-5fe8dacfc998 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:17.476205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c4376-dbb8-4c38-b2f9-5fe8dacfc998 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:17.476672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c4376-dbb8-4c38-b2f9-5fe8dacfc998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:17.498144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-fb4c4376-dbb8-4c38-b2f9-5fe8dacfc998 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:17.499805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-fb4c4376-dbb8-4c38-b2f9-5fe8dacfc998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:17.584572 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdec72-b35c-4b6a-8f15-ce9c44fda123 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:17.597649 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 429/868] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:11:17.620857 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c4376-dbb8-4c38-b2f9-5fe8dacfc998 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:17.628884 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c692e-6f90-4d4e-b1a3-c3c9a00c4d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:17.629927 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c692e-6f90-4d4e-b1a3-c3c9a00c4d35 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:17.630409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c692e-6f90-4d4e-b1a3-c3c9a00c4d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:17.630883 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c692e-6f90-4d4e-b1a3-c3c9a00c4d35 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:17.631791 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c692e-6f90-4d4e-b1a3-c3c9a00c4d35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:17.632328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c692e-6f90-4d4e-b1a3-c3c9a00c4d35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:17.633944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c692e-6f90-4d4e-b1a3-c3c9a00c4d35 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:17.646925 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 440/869] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:17.653814 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c692e-6f90-4d4e-b1a3-c3c9a00c4d35 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:17.698882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:17.744751 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c4376-dbb8-4c38-b2f9-5fe8dacfc998 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:17.762911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:17.783350 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d5fef-7c87-4974-bcbe-d564d735172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:17.784440 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d5fef-7c87-4974-bcbe-d564d735172f None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:17.784942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d5fef-7c87-4974-bcbe-d564d735172f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:17.785441 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d5fef-7c87-4974-bcbe-d564d735172f None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:17.786453 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d5fef-7c87-4974-bcbe-d564d735172f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:17.788052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d5fef-7c87-4974-bcbe-d564d735172f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:17.794000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c692e-6f90-4d4e-b1a3-c3c9a00c4d35 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:17.810070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 430/870] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 15:11:17 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:11:17.845830 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd6f80-b34a-4ac4-8e40-81a1f7f785a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:17.846561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd6f80-b34a-4ac4-8e40-81a1f7f785a4 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:17.846895 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd6f80-b34a-4ac4-8e40-81a1f7f785a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:17.847228 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd6f80-b34a-4ac4-8e40-81a1f7f785a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:17.855411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd6f80-b34a-4ac4-8e40-81a1f7f785a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:17.857316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd6f80-b34a-4ac4-8e40-81a1f7f785a4 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:17.870737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-a6dd6f80-b34a-4ac4-8e40-81a1f7f785a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:17.870737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:17.870737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:17.870737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:17.870737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:17.870737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:17.870737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:17.870737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:17.870737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:17.870737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:17.870737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:17.870737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:17.870737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:11:17.870737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:11:17.870737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:17.871711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:17.872613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:11:17.872613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:11:17.872613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:11:17.872613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:11:17.872613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:11:17.872613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:11:17.872613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:17.872613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:17.872613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 441/871] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:17.878553 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d5fef-7c87-4974-bcbe-d564d735172f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:18.000104 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d5fef-7c87-4974-bcbe-d564d735172f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:18.036962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a8d71-84bc-4ca2-93a6-4834b4e3da91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:18.037968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a8d71-84bc-4ca2-93a6-4834b4e3da91 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:18.038456 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a8d71-84bc-4ca2-93a6-4834b4e3da91 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:18.038918 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a8d71-84bc-4ca2-93a6-4834b4e3da91 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:18.039864 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a8d71-84bc-4ca2-93a6-4834b4e3da91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:18.040359 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a8d71-84bc-4ca2-93a6-4834b4e3da91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:18.042043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a8d71-84bc-4ca2-93a6-4834b4e3da91 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:18.055721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 431/872] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:11:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:18.061543 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd6f80-b34a-4ac4-8e40-81a1f7f785a4 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:18.154579 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:18.170150 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd6f80-b34a-4ac4-8e40-81a1f7f785a4 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:18.208114 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f3418-b93b-4a4b-998d-d54013178fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:18.208754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f3418-b93b-4a4b-998d-d54013178fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:18.209033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f3418-b93b-4a4b-998d-d54013178fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:18.209306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f3418-b93b-4a4b-998d-d54013178fa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:18.210156 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f3418-b93b-4a4b-998d-d54013178fa6 None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=25a26f54accf4264aef646c3757de7e9) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:18.210474 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f3418-b93b-4a4b-998d-d54013178fa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:18.211924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6f3418-b93b-4a4b-998d-d54013178fa6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:18.219402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 442/873] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:11:18 2022] GET /identity/v3/role_assignments?user.id=25a26f54accf4264aef646c3757de7e9&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:18.319683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f36c5d-0fae-4807-b27f-cd3b04613e85 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:18.320210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f36c5d-0fae-4807-b27f-cd3b04613e85 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:18.320210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f36c5d-0fae-4807-b27f-cd3b04613e85 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:18.321878 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 432/874] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:18.327873 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a3865-9a28-4cc0-bd9e-21245f5ed7a0 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:18.328382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a3865-9a28-4cc0-bd9e-21245f5ed7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:18.328459 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a3865-9a28-4cc0-bd9e-21245f5ed7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:18.349358 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-c00a3865-9a28-4cc0-bd9e-21245f5ed7a0 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:18.350620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c00a3865-9a28-4cc0-bd9e-21245f5ed7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:18.427957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 443/875] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:18 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:18.451226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f36c5d-0fae-4807-b27f-cd3b04613e85 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:18.492306 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:18.563753 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f36c5d-0fae-4807-b27f-cd3b04613e85 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:18.602719 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae836a4-a7eb-4666-8b49-ccf31a66d2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:18.603378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae836a4-a7eb-4666-8b49-ccf31a66d2b3 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:18.603660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae836a4-a7eb-4666-8b49-ccf31a66d2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:18.603937 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae836a4-a7eb-4666-8b49-ccf31a66d2b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:18.615949 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae836a4-a7eb-4666-8b49-ccf31a66d2b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:18.621098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-3ae836a4-a7eb-4666-8b49-ccf31a66d2b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:11:18.621098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:18.621098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:18.621098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:18.621098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:18.621098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:18.621098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:18.621098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:18.621098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:18.621098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:18.621098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:18.621098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:18.621098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:18.621098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:18.621098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:18.621098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:18.622052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:18.622052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:18.622052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:18.622052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:18.622052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:18.622052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:18.622052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:18.622052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:18.622052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:18.622052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:18.622052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:18.622052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:18.622052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:18.622052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:18.622052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:18.622052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:18.623731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:18.623731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:18.623731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:18.623731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:18.623731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:18.623731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:18.623731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:18.623731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:18.623731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:18.623731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:18.623731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:11:18.623731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:18.623731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 433/876] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 15:11:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:18.628785 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a3865-9a28-4cc0-bd9e-21245f5ed7a0 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:18.713257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:18.732446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a3865-9a28-4cc0-bd9e-21245f5ed7a0 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:18.772117 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f65198-3793-491f-81e7-66bef1f97235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:18.772759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f65198-3793-491f-81e7-66bef1f97235 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:18.773040 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f65198-3793-491f-81e7-66bef1f97235 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:18.773375 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f65198-3793-491f-81e7-66bef1f97235 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:18.774146 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f65198-3793-491f-81e7-66bef1f97235 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:18.774500 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f65198-3793-491f-81e7-66bef1f97235 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:18.775950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f65198-3793-491f-81e7-66bef1f97235 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:18.785328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 444/877] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 15:11:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:18.790737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae836a4-a7eb-4666-8b49-ccf31a66d2b3 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:18.893030 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:18.899922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae836a4-a7eb-4666-8b49-ccf31a66d2b3 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:18.935736 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30e61b-9430-4783-a93d-f0db86bb09ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:18.936388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30e61b-9430-4783-a93d-f0db86bb09ef None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:18.936668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30e61b-9430-4783-a93d-f0db86bb09ef None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:18.936963 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30e61b-9430-4783-a93d-f0db86bb09ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:18.947581 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30e61b-9430-4783-a93d-f0db86bb09ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:18.949071 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30e61b-9430-4783-a93d-f0db86bb09ef None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:18.965463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-bc30e61b-9430-4783-a93d-f0db86bb09ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:18.965463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:18.965463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:18.965463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:18.965463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:18.965463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:18.965463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:18.965463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:18.965463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:18.965463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:18.965463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:18.965463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:18.965463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:18.965463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:18.965463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:18.967583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:18.969873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:18.969873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:18.969873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:18.969873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:18.969873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 434/878] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:18.972800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f65198-3793-491f-81e7-66bef1f97235 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:19.080575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f65198-3793-491f-81e7-66bef1f97235 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:19.104303 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cbfc6-3e7b-4f1c-bffa-26904f7f7d05 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:19.104644 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cbfc6-3e7b-4f1c-bffa-26904f7f7d05 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:19.108050 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cbfc6-3e7b-4f1c-bffa-26904f7f7d05 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:19.108050 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 435/879] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:11:19.112270 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279b800-a4b0-491f-aff7-ec35d019be6a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:19.112559 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279b800-a4b0-491f-aff7-ec35d019be6a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:19.112837 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279b800-a4b0-491f-aff7-ec35d019be6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:19.119571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de216d-bd89-46e0-bb69-1316f5725de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:19.120570 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de216d-bd89-46e0-bb69-1316f5725de5 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:19.121069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de216d-bd89-46e0-bb69-1316f5725de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:19.121551 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de216d-bd89-46e0-bb69-1316f5725de5 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:19.122484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de216d-bd89-46e0-bb69-1316f5725de5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:19.122994 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de216d-bd89-46e0-bb69-1316f5725de5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:19.124687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de216d-bd89-46e0-bb69-1316f5725de5 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:19.133032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-8279b800-a4b0-491f-aff7-ec35d019be6a None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:19.134696 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8279b800-a4b0-491f-aff7-ec35d019be6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:19.138289 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 445/880] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:19.147309 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de216d-bd89-46e0-bb69-1316f5725de5 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:19.151798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:19.214073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 436/881] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:19 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:11:19.229953 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279b800-a4b0-491f-aff7-ec35d019be6a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:19.279096 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de216d-bd89-46e0-bb69-1316f5725de5 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:19.315362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:19.326029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01eb54-5b5c-4613-8b2e-d34d9f59d7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:19.326682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01eb54-5b5c-4613-8b2e-d34d9f59d7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:19.326954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01eb54-5b5c-4613-8b2e-d34d9f59d7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:19.327222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01eb54-5b5c-4613-8b2e-d34d9f59d7e7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:19.338077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01eb54-5b5c-4613-8b2e-d34d9f59d7e7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:19.339567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01eb54-5b5c-4613-8b2e-d34d9f59d7e7 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:19.344480 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279b800-a4b0-491f-aff7-ec35d019be6a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:19.352519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-ae01eb54-5b5c-4613-8b2e-d34d9f59d7e7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:11:19.352519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:19.352519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:19.352519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:19.352519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:19.352519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:19.352519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:19.352519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:19.352519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:19.352519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:19.352519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:19.352519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:19.352519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:11:19.352519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:11:19.352519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:11:19.352519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:19.353695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:19.354686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:19.354686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:19.354686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:11:19.354686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:11:19.354686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:11:19.354686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:11:19.354686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:11:19.354686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:11:19.354686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:11:19.354686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:19.354686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 446/882] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 15:11:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:19.359896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01eb54-5b5c-4613-8b2e-d34d9f59d7e7 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:19.391464 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7256023-5c2b-454f-b772-cdb031b70117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:19.392697 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7256023-5c2b-454f-b772-cdb031b70117 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:19.393556 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7256023-5c2b-454f-b772-cdb031b70117 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:19.394294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7256023-5c2b-454f-b772-cdb031b70117 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:19.395191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7256023-5c2b-454f-b772-cdb031b70117 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:19.397942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7256023-5c2b-454f-b772-cdb031b70117 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:19.418910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 437/883] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 15:11:19 2022] POST /identity/v3/services => generated 269 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:11:19.429183 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:19.480211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01eb54-5b5c-4613-8b2e-d34d9f59d7e7 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:19.531589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107ff4d-8d99-4865-8a64-466550413fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:19.532790 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107ff4d-8d99-4865-8a64-466550413fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:19.533316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107ff4d-8d99-4865-8a64-466550413fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:19.533911 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107ff4d-8d99-4865-8a64-466550413fd3 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:19.535020 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107ff4d-8d99-4865-8a64-466550413fd3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:19.535555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107ff4d-8d99-4865-8a64-466550413fd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:19.537431 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107ff4d-8d99-4865-8a64-466550413fd3 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:19.560459 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 447/884] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 15:11:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:19.566138 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7256023-5c2b-454f-b772-cdb031b70117 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:19.678800 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7256023-5c2b-454f-b772-cdb031b70117 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:19.713181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb02c7-c1c5-4406-8bf8-90818d5725a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:19.713832 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb02c7-c1c5-4406-8bf8-90818d5725a1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:19.714112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb02c7-c1c5-4406-8bf8-90818d5725a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:19.714381 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb02c7-c1c5-4406-8bf8-90818d5725a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:19.723506 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:19.787685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb02c7-c1c5-4406-8bf8-90818d5725a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:19.789161 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb02c7-c1c5-4406-8bf8-90818d5725a1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:19.903597 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 438/885] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:11:19 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:11:19.909290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107ff4d-8d99-4865-8a64-466550413fd3 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:19.966296 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac295bde-7038-4450-b050-3312190050d0 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:19.966962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac295bde-7038-4450-b050-3312190050d0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:19.967462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac295bde-7038-4450-b050-3312190050d0 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:19.969085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 439/886] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:11:19.974519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26408add-add6-41b8-9b4c-92916e61e068 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:19.975167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26408add-add6-41b8-9b4c-92916e61e068 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:19.975653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26408add-add6-41b8-9b4c-92916e61e068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:19.995446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-26408add-add6-41b8-9b4c-92916e61e068 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:19.997009 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-26408add-add6-41b8-9b4c-92916e61e068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:20.030497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:20.033406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107ff4d-8d99-4865-8a64-466550413fd3 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:20.058091 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:20.079331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 440/887] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:11:20.087083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e34cdb-74f6-4dce-a0e0-e9dd9b2133b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:20.093821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e34cdb-74f6-4dce-a0e0-e9dd9b2133b4 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:20.093821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e34cdb-74f6-4dce-a0e0-e9dd9b2133b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:20.093821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e34cdb-74f6-4dce-a0e0-e9dd9b2133b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:20.103407 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e34cdb-74f6-4dce-a0e0-e9dd9b2133b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:20.105211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e34cdb-74f6-4dce-a0e0-e9dd9b2133b4 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:20.105697 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26408add-add6-41b8-9b4c-92916e61e068 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:20.122721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-c3e34cdb-74f6-4dce-a0e0-e9dd9b2133b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:20.122721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:20.122721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:20.122721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:20.122721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:20.122721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:20.122721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:20.122721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:20.122721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:20.122721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:20.122721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:20.122721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:20.122721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:20.122721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:20.122721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:20.123768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:20.124769 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:20.124769 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:20.124769 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:20.124769 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:20.124769 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 448/888] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:20.132345 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e34cdb-74f6-4dce-a0e0-e9dd9b2133b4 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:20.232501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26408add-add6-41b8-9b4c-92916e61e068 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:20.261316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e34cdb-74f6-4dce-a0e0-e9dd9b2133b4 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:20.272369 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201401d-ecec-4bb9-a43c-f8126e9b6b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:20.273132 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201401d-ecec-4bb9-a43c-f8126e9b6b5a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:20.273700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201401d-ecec-4bb9-a43c-f8126e9b6b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:20.273856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201401d-ecec-4bb9-a43c-f8126e9b6b5a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:20.274701 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201401d-ecec-4bb9-a43c-f8126e9b6b5a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:20.276734 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201401d-ecec-4bb9-a43c-f8126e9b6b5a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-1201401d-ecec-4bb9-a43c-f8126e9b6b5a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:20.287229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:20.288288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:20.288288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:20.288288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:20.288288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:20.288288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:20.288288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:20.288288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:20.288288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:20.288288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:20.288288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:20.288288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:20.288288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:20.288288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:20.288288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:20.288288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:20.288288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:20.289302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:20.289302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:20.289302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:20.289302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:20.289302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:20.289302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:20.289302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:20.289302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:20.289302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:20.289302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:20.289302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:20.289302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:20.289302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:11:20.289302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:20.289302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 441/889] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:11:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:20.294349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201401d-ecec-4bb9-a43c-f8126e9b6b5a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:20.302678 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa85251-8d6a-4687-bd53-80be1e863842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:20.303493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa85251-8d6a-4687-bd53-80be1e863842 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:20.303930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa85251-8d6a-4687-bd53-80be1e863842 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:20.304346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa85251-8d6a-4687-bd53-80be1e863842 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:20.305217 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa85251-8d6a-4687-bd53-80be1e863842 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:20.305689 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa85251-8d6a-4687-bd53-80be1e863842 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:20.307305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa85251-8d6a-4687-bd53-80be1e863842 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:20.324879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 449/890] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:20.326617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa85251-8d6a-4687-bd53-80be1e863842 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:20.328989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:20.359250 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:20.408832 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201401d-ecec-4bb9-a43c-f8126e9b6b5a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:20.433006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa85251-8d6a-4687-bd53-80be1e863842 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:20.447082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69300e88-a05b-4a83-905a-8a984a234fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:20.447811 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69300e88-a05b-4a83-905a-8a984a234fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:20.448162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69300e88-a05b-4a83-905a-8a984a234fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:20.448517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69300e88-a05b-4a83-905a-8a984a234fa2 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:20.449421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69300e88-a05b-4a83-905a-8a984a234fa2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:20.449804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69300e88-a05b-4a83-905a-8a984a234fa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:20.451419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69300e88-a05b-4a83-905a-8a984a234fa2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:20.457146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 442/891] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:11:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:20.462366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69300e88-a05b-4a83-905a-8a984a234fa2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:20.471753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac46793-41b8-41e2-8ca3-1eb056afcff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:20.472749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac46793-41b8-41e2-8ca3-1eb056afcff0 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:20.473254 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac46793-41b8-41e2-8ca3-1eb056afcff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:20.473747 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac46793-41b8-41e2-8ca3-1eb056afcff0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:20.481321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac46793-41b8-41e2-8ca3-1eb056afcff0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:20.483435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac46793-41b8-41e2-8ca3-1eb056afcff0 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:20.495408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-8ac46793-41b8-41e2-8ca3-1eb056afcff0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:20.495408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:20.495408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:20.495408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:20.495408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:20.495408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:20.495408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:20.495408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:20.495408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:20.495408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:20.495408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:20.495408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:20.495408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:11:20.495408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:11:20.495408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:20.496378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:20.497271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:11:20.497271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:11:20.497271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:11:20.497271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:11:20.497271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:11:20.497271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:11:20.497271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:20.497271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:20.504273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 450/892] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:20.504273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496601ec-c8dc-48f0-8097-efd70e5838b8 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:20.504273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496601ec-c8dc-48f0-8097-efd70e5838b8 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:20.504273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496601ec-c8dc-48f0-8097-efd70e5838b8 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:20.505404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 451/893] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:20.506004 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496601ec-c8dc-48f0-8097-efd70e5838b8 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:20.565436 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:20.572573 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69300e88-a05b-4a83-905a-8a984a234fa2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:20.608672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:20.612600 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e663282-a18a-4ff4-b927-73b146f4f3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:20.613265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e663282-a18a-4ff4-b927-73b146f4f3bf None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:20.613543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e663282-a18a-4ff4-b927-73b146f4f3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:20.613845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e663282-a18a-4ff4-b927-73b146f4f3bf None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:20.614551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e663282-a18a-4ff4-b927-73b146f4f3bf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:20.614860 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e663282-a18a-4ff4-b927-73b146f4f3bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:20.616301 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e663282-a18a-4ff4-b927-73b146f4f3bf None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:20.621856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 443/894] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:11:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:20.622092 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496601ec-c8dc-48f0-8097-efd70e5838b8 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:20.623170 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14968d3-ed9d-4881-b059-97339f7364b0 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:20.623462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14968d3-ed9d-4881-b059-97339f7364b0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:20.623738 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14968d3-ed9d-4881-b059-97339f7364b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:20.642883 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b14968d3-ed9d-4881-b059-97339f7364b0 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:20.644124 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b14968d3-ed9d-4881-b059-97339f7364b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:20.664124 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a721ed-fe65-4fba-b59f-1b8f24e13067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:20.664756 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a721ed-fe65-4fba-b59f-1b8f24e13067 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:20.665028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a721ed-fe65-4fba-b59f-1b8f24e13067 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:20.665576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a721ed-fe65-4fba-b59f-1b8f24e13067 None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:20.666072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a721ed-fe65-4fba-b59f-1b8f24e13067 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:20.666373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a721ed-fe65-4fba-b59f-1b8f24e13067 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:20.667810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a721ed-fe65-4fba-b59f-1b8f24e13067 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:20.680110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 452/895] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:11:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:20.680899 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a721ed-fe65-4fba-b59f-1b8f24e13067 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:20.719517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 444/896] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:20.720484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14968d3-ed9d-4881-b059-97339f7364b0 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:20.795431 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a721ed-fe65-4fba-b59f-1b8f24e13067 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:20.828216 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:20.834780 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac998e80-5787-41f2-aa2e-ea309b775f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:20.835419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac998e80-5787-41f2-aa2e-ea309b775f9c None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:20.835686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac998e80-5787-41f2-aa2e-ea309b775f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:20.835974 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac998e80-5787-41f2-aa2e-ea309b775f9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:20.836731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac998e80-5787-41f2-aa2e-ea309b775f9c None admin] RBAC: Adding query filter params (interface=public, service_id=a57727dd2cb8464abeea80c2e445f1c1, region_id=RegionOne) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:20.837044 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac998e80-5787-41f2-aa2e-ea309b775f9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:20.838459 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac998e80-5787-41f2-aa2e-ea309b775f9c None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:20.839741 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14968d3-ed9d-4881-b059-97339f7364b0 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:20.850633 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 453/897] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 15:11:20 2022] GET /identity/v3/endpoints?service_id=a57727dd2cb8464abeea80c2e445f1c1&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:20.850633 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac998e80-5787-41f2-aa2e-ea309b775f9c None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:20.871510 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:20.881045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ed8df-e073-4c81-9201-3110e91a67a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:20.881683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ed8df-e073-4c81-9201-3110e91a67a7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:20.881980 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ed8df-e073-4c81-9201-3110e91a67a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:20.882250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ed8df-e073-4c81-9201-3110e91a67a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:20.883055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ed8df-e073-4c81-9201-3110e91a67a7 None admin] RBAC: Adding query filter params (role.id=8d5e26281fb04bc68fc31532f2fb3a8d, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=38b18e5d1e8843c6a08b1f5609cc60d5) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:20.883526 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ed8df-e073-4c81-9201-3110e91a67a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:20.884811 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ed8df-e073-4c81-9201-3110e91a67a7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:20.890715 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 445/898] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:11:20 2022] GET /identity/v3/role_assignments?user.id=38b18e5d1e8843c6a08b1f5609cc60d5&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=8d5e26281fb04bc68fc31532f2fb3a8d => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:20.891490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ed8df-e073-4c81-9201-3110e91a67a7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:20.982697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac998e80-5787-41f2-aa2e-ea309b775f9c None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:21.026276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76dd03-ae1f-4a02-895d-64b6bbe41c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:21.027255 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76dd03-ae1f-4a02-895d-64b6bbe41c34 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:21.027687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76dd03-ae1f-4a02-895d-64b6bbe41c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:21.028132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76dd03-ae1f-4a02-895d-64b6bbe41c34 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:21.028970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76dd03-ae1f-4a02-895d-64b6bbe41c34 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:21.030624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76dd03-ae1f-4a02-895d-64b6bbe41c34 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:21.034721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ed8df-e073-4c81-9201-3110e91a67a7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-9e76dd03-ae1f-4a02-895d-64b6bbe41c34 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:21.041251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:21.042400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:21.042400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:21.042400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:21.042400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:21.042400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:21.042400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:21.042400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:21.042400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:21.042400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:21.042400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:21.042400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:21.042400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:21.042400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:21.042400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:21.042400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:21.042400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:21.043259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:21.043259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:21.043259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:21.043259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:21.043259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:21.043259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:21.043259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:21.043259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:21.043259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:21.043259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:21.043259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:21.043259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:21.043259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:11:21.043259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:21.046174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 454/899] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 15:11:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:21.056841 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76dd03-ae1f-4a02-895d-64b6bbe41c34 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:21.094144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c00a6b-df21-4a41-ac02-fd6afcbc2256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:21.095384 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c00a6b-df21-4a41-ac02-fd6afcbc2256 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:21.096701 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c00a6b-df21-4a41-ac02-fd6afcbc2256 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:21.097187 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c00a6b-df21-4a41-ac02-fd6afcbc2256 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:21.098112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c00a6b-df21-4a41-ac02-fd6afcbc2256 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:21.099719 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c00a6b-df21-4a41-ac02-fd6afcbc2256 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:21.109158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 446/900] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 15:11:20 2022] GET /identity/v3/services => generated 1268 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:11:21.166566 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:21.176746 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76dd03-ae1f-4a02-895d-64b6bbe41c34 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:21.216714 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b862da-80cd-4019-ad13-8e67082ae5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:21.217751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b862da-80cd-4019-ad13-8e67082ae5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:21.218356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b862da-80cd-4019-ad13-8e67082ae5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:21.218846 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b862da-80cd-4019-ad13-8e67082ae5b6 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:21.219851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b862da-80cd-4019-ad13-8e67082ae5b6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:21.220368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b862da-80cd-4019-ad13-8e67082ae5b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:21.222090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b862da-80cd-4019-ad13-8e67082ae5b6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:21.228294 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 455/901] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 15:11:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:21.235148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c00a6b-df21-4a41-ac02-fd6afcbc2256 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:21.261912 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:21.349701 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c00a6b-df21-4a41-ac02-fd6afcbc2256 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:21.385582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57f2ed-ba94-48aa-90c3-563cf688e69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:21.386338 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57f2ed-ba94-48aa-90c3-563cf688e69c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:21.386618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57f2ed-ba94-48aa-90c3-563cf688e69c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:21.386888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57f2ed-ba94-48aa-90c3-563cf688e69c None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:21.387583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57f2ed-ba94-48aa-90c3-563cf688e69c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:21.387882 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57f2ed-ba94-48aa-90c3-563cf688e69c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:21.389273 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57f2ed-ba94-48aa-90c3-563cf688e69c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:21.394756 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 447/902] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 15:11:21 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:21.536492 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ba20f-0123-4bfb-a2b6-06b8f7d5b1a8 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:21.537041 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ba20f-0123-4bfb-a2b6-06b8f7d5b1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:21.537041 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ba20f-0123-4bfb-a2b6-06b8f7d5b1a8 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:21.538723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 456/903] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:21.544597 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90585a08-ca6c-4cd9-86fc-f0384bf7a2d9 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:21.545134 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90585a08-ca6c-4cd9-86fc-f0384bf7a2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:21.545134 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90585a08-ca6c-4cd9-86fc-f0384bf7a2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:21.564920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-90585a08-ca6c-4cd9-86fc-f0384bf7a2d9 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:21.566156 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-90585a08-ca6c-4cd9-86fc-f0384bf7a2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:21.643488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 448/904] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:21.665716 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ba20f-0123-4bfb-a2b6-06b8f7d5b1a8 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:21.778697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ba20f-0123-4bfb-a2b6-06b8f7d5b1a8 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:21.813288 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbac0b-7a46-4679-be88-0413d610117e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:21.814055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbac0b-7a46-4679-be88-0413d610117e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:21.814339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbac0b-7a46-4679-be88-0413d610117e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:21.814612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbac0b-7a46-4679-be88-0413d610117e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:21.815247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbac0b-7a46-4679-be88-0413d610117e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:21.816618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbac0b-7a46-4679-be88-0413d610117e None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:21.825958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-fcfbac0b-7a46-4679-be88-0413d610117e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:11:21.825958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:21.825958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:21.825958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:21.825958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:21.825958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:21.825958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:21.825958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:21.825958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:11:21.825958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:21.825958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:21.825958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:21.825958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:21.825958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:21.825958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:21.825958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:21.826937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:21.826937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:21.826937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:21.826937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:21.826937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:21.826937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:21.826937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:21.826937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:21.826937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:21.826937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:21.826937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:21.826937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:21.826937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:21.826937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:21.826937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:21.826937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:21.827993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:21.827993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:21.827993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:21.827993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:21.827993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:21.827993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:21.827993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:21.827993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:21.827993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:21.827993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:21.827993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:21.827993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:21.827993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:21.827993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:11:21.827993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:21.827993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 457/905] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Aug 1 15:11:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:21.835653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90585a08-ca6c-4cd9-86fc-f0384bf7a2d9 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:21.858410 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:21.945737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90585a08-ca6c-4cd9-86fc-f0384bf7a2d9 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:21.982699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74544f-b368-4264-8073-d420b1d3eb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:21.983337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74544f-b368-4264-8073-d420b1d3eb63 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:21.983618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74544f-b368-4264-8073-d420b1d3eb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:21.983964 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74544f-b368-4264-8073-d420b1d3eb63 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:21.984656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74544f-b368-4264-8073-d420b1d3eb63 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:21.984958 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74544f-b368-4264-8073-d420b1d3eb63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:21.986388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74544f-b368-4264-8073-d420b1d3eb63 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:21.991726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 449/906] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 15:11:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:21.998923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbac0b-7a46-4679-be88-0413d610117e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:22.011601 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:22.112547 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbac0b-7a46-4679-be88-0413d610117e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:22.149968 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c482c-cc07-4f35-9f3e-ffe972e0e8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:22.150659 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c482c-cc07-4f35-9f3e-ffe972e0e8ab None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:22.150932 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c482c-cc07-4f35-9f3e-ffe972e0e8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:22.151202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c482c-cc07-4f35-9f3e-ffe972e0e8ab None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:22.151894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c482c-cc07-4f35-9f3e-ffe972e0e8ab None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:22.152193 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c482c-cc07-4f35-9f3e-ffe972e0e8ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:22.153573 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c482c-cc07-4f35-9f3e-ffe972e0e8ab None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:22.159492 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 458/907] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 15:11:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:22.165151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74544f-b368-4264-8073-d420b1d3eb63 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:22.261357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:22.288826 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74544f-b368-4264-8073-d420b1d3eb63 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:22.332814 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ac13f-06d6-47f7-a868-b02c3b98298e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:22.333521 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ac13f-06d6-47f7-a868-b02c3b98298e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:22.333823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ac13f-06d6-47f7-a868-b02c3b98298e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:22.334103 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ac13f-06d6-47f7-a868-b02c3b98298e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:22.334881 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ac13f-06d6-47f7-a868-b02c3b98298e None admin] RBAC: Adding query filter params (interface=public, service_id=cbbd4cccfed84b22a83eeb103564f2be, region_id=RegionOne) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:22.335187 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ac13f-06d6-47f7-a868-b02c3b98298e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:22.336587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ac13f-06d6-47f7-a868-b02c3b98298e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:22.342902 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 450/908] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 15:11:22 2022] GET /identity/v3/endpoints?service_id=cbbd4cccfed84b22a83eeb103564f2be&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:22.348773 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c482c-cc07-4f35-9f3e-ffe972e0e8ab None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:22.425609 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:22.459049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c482c-cc07-4f35-9f3e-ffe972e0e8ab None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:22.495216 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1514a-6552-4147-b2bf-931b09b8f557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:22.495854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1514a-6552-4147-b2bf-931b09b8f557 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:22.496135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1514a-6552-4147-b2bf-931b09b8f557 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:22.496407 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1514a-6552-4147-b2bf-931b09b8f557 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:22.497072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1514a-6552-4147-b2bf-931b09b8f557 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:22.498490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1514a-6552-4147-b2bf-931b09b8f557 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:22.505299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 459/909] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 15:11:22 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:11:23.035824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111be91-bbff-45f9-a7ae-0f95a43b05cb None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:23.036613 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111be91-bbff-45f9-a7ae-0f95a43b05cb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:23.036613 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111be91-bbff-45f9-a7ae-0f95a43b05cb None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:23.038332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 451/910] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:23.045503 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581ba53-4959-44af-b12c-28a0da32fc9d None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:23.046568 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581ba53-4959-44af-b12c-28a0da32fc9d None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:23.046568 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581ba53-4959-44af-b12c-28a0da32fc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:23.069278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-9581ba53-4959-44af-b12c-28a0da32fc9d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:23.070498 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9581ba53-4959-44af-b12c-28a0da32fc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:23.145973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 460/911] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:23.162709 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111be91-bbff-45f9-a7ae-0f95a43b05cb None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:23.270338 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111be91-bbff-45f9-a7ae-0f95a43b05cb None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:23.304713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfc8ad-573f-4640-b09e-c76a5194e882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:23.305455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfc8ad-573f-4640-b09e-c76a5194e882 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:23.306440 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfc8ad-573f-4640-b09e-c76a5194e882 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:23.306861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfc8ad-573f-4640-b09e-c76a5194e882 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:23.307312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c168c-8daa-42fb-8596-49463e860f0c None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:23.307609 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c168c-8daa-42fb-8596-49463e860f0c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:23.307879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c168c-8daa-42fb-8596-49463e860f0c None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:23.313203 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 461/912] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:11:23.314763 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2ec59-9369-43e5-8159-153af7fefb53 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:23.315042 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2ec59-9369-43e5-8159-153af7fefb53 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:23.315311 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2ec59-9369-43e5-8159-153af7fefb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:23.318266 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfc8ad-573f-4640-b09e-c76a5194e882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:23.323403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-a0bfc8ad-573f-4640-b09e-c76a5194e882 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:11:23.323403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:23.323403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:23.323403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:23.323403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:23.323403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:23.323403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:23.323403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:23.323403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:23.323403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:23.323403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:23.323403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:23.323403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:23.323403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:23.323403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:23.323403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:23.325546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:23.325546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:23.325546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:23.325546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:23.325546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:23.325546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:23.325546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:23.325546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:23.325546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:23.325546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:23.325546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:23.325546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:23.325546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:23.325546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:23.325546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:23.325546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:23.327582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:23.327582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:23.327582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:23.327582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:23.327582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:23.327582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:23.327582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:23.327582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:23.327582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:23.327582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:23.327582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:11:23.327582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:23.327582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 452/913] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 15:11:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:23.327582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:23.335054 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfc8ad-573f-4640-b09e-c76a5194e882 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:23.339237 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-6ed2ec59-9369-43e5-8159-153af7fefb53 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:23.340481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6ed2ec59-9369-43e5-8159-153af7fefb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:23.401677 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:23.415267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 462/914] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:23.435903 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2ec59-9369-43e5-8159-153af7fefb53 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:23.460330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfc8ad-573f-4640-b09e-c76a5194e882 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:23.498659 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33da5b-c7d4-4787-b738-f1eb30b6b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:23.499787 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33da5b-c7d4-4787-b738-f1eb30b6b2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:23.500354 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33da5b-c7d4-4787-b738-f1eb30b6b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:23.500832 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33da5b-c7d4-4787-b738-f1eb30b6b2f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:23.501876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33da5b-c7d4-4787-b738-f1eb30b6b2f2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:23.502421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33da5b-c7d4-4787-b738-f1eb30b6b2f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:23.504194 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33da5b-c7d4-4787-b738-f1eb30b6b2f2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:23.513539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 453/915] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 15:11:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:23.520526 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee3dcd-9444-4769-bcd0-b796dc7a1787 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:23.520809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee3dcd-9444-4769-bcd0-b796dc7a1787 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:23.521094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee3dcd-9444-4769-bcd0-b796dc7a1787 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:23.522539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 454/916] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:11:23.523100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee3dcd-9444-4769-bcd0-b796dc7a1787 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:23.554749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2ec59-9369-43e5-8159-153af7fefb53 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:23.591778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:23.592293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f5e92-1769-43d7-b629-4c0a3b16160f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:23.593033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f5e92-1769-43d7-b629-4c0a3b16160f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:23.593771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f5e92-1769-43d7-b629-4c0a3b16160f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:23.594153 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f5e92-1769-43d7-b629-4c0a3b16160f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:23.594776 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f5e92-1769-43d7-b629-4c0a3b16160f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:23.596148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f5e92-1769-43d7-b629-4c0a3b16160f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-e78f5e92-1769-43d7-b629-4c0a3b16160f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:23.606404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:23.607799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:23.607799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:23.607799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:23.607799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:23.607799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:23.607799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:23.607799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:23.607799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:23.607799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:23.607799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:23.607799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:23.607799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:23.607799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:23.607799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:23.607799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:23.607799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:23.608700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:23.608700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:23.608700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:23.608700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:23.608700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:23.608700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:23.608700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:23.608700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:23.608700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:23.608700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:23.608700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:23.608700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:23.608700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:11:23.608700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:23.608700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 463/917] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:11:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:23.609521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb9d58-f4d0-42b8-869c-2d533949819b None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:23.609521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb9d58-f4d0-42b8-869c-2d533949819b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:23.610607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb9d58-f4d0-42b8-869c-2d533949819b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:23.628652 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-60eb9d58-f4d0-42b8-869c-2d533949819b None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:23.629811 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-60eb9d58-f4d0-42b8-869c-2d533949819b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:23.642740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee3dcd-9444-4769-bcd0-b796dc7a1787 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:23.684165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f25993-0eb7-4889-bb63-dd322d8560f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:23.684759 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f25993-0eb7-4889-bb63-dd322d8560f6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:23.685041 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f25993-0eb7-4889-bb63-dd322d8560f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:23.685317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f25993-0eb7-4889-bb63-dd322d8560f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:23.691770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:23.698647 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f25993-0eb7-4889-bb63-dd322d8560f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:23.700138 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f25993-0eb7-4889-bb63-dd322d8560f6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:23.707918 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 464/918] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:23.708502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb9d58-f4d0-42b8-869c-2d533949819b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:23.728828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-47f25993-0eb7-4889-bb63-dd322d8560f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:23.728828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:23.728828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:23.728828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:23.728828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:23.728828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:23.728828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:23.728828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:23.728828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:23.728828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:23.728828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:23.728828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:23.728828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:23.728828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:23.728828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:23.729819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:23.730718 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:23.730718 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:23.730718 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:23.730718 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:23.730718 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 455/919] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:23.733247 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f25993-0eb7-4889-bb63-dd322d8560f6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:23.821603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb9d58-f4d0-42b8-869c-2d533949819b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:23.841847 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f25993-0eb7-4889-bb63-dd322d8560f6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:23.857107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfd10f-3331-4097-8191-5d5ded680104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:23.858099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:23.858616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfd10f-3331-4097-8191-5d5ded680104 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:23.859267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfd10f-3331-4097-8191-5d5ded680104 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:23.859710 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfd10f-3331-4097-8191-5d5ded680104 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:23.860523 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfd10f-3331-4097-8191-5d5ded680104 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:23.864448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfd10f-3331-4097-8191-5d5ded680104 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:23.865916 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfd10f-3331-4097-8191-5d5ded680104 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:23.873934 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 465/920] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:11:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:23.875066 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfd10f-3331-4097-8191-5d5ded680104 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:23.887412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1be01-a08c-4797-9f79-78c7f2defbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:23.888057 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1be01-a08c-4797-9f79-78c7f2defbbe None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:23.888342 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1be01-a08c-4797-9f79-78c7f2defbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:23.888615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1be01-a08c-4797-9f79-78c7f2defbbe None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:23.889325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1be01-a08c-4797-9f79-78c7f2defbbe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:23.890802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1be01-a08c-4797-9f79-78c7f2defbbe None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:23.906856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 456/921] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 15:11:23 2022] POST /identity/v3/services => generated 241 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:11:23.907518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1be01-a08c-4797-9f79-78c7f2defbbe None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:23.946360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:24.011102 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfd10f-3331-4097-8191-5d5ded680104 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:24.041425 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1be01-a08c-4797-9f79-78c7f2defbbe None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:24.056482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ed53d-d17e-48d2-9257-562b252dab66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:24.057694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed53d-d17e-48d2-9257-562b252dab66 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:24.058142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed53d-d17e-48d2-9257-562b252dab66 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:24.058487 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed53d-d17e-48d2-9257-562b252dab66 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:24.059893 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ed53d-d17e-48d2-9257-562b252dab66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:24.060297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ed53d-d17e-48d2-9257-562b252dab66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:24.062230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ed53d-d17e-48d2-9257-562b252dab66 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:24.081840 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 466/922] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:24.085953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ed53d-d17e-48d2-9257-562b252dab66 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:24.090812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bd83d-a237-4818-a6cb-b6287dde0753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:24.091601 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bd83d-a237-4818-a6cb-b6287dde0753 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:24.091943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bd83d-a237-4818-a6cb-b6287dde0753 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:24.092282 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bd83d-a237-4818-a6cb-b6287dde0753 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:24.093224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bd83d-a237-4818-a6cb-b6287dde0753 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:24.093609 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bd83d-a237-4818-a6cb-b6287dde0753 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:24.095213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bd83d-a237-4818-a6cb-b6287dde0753 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:24.102882 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 457/923] 10.222.0.48 () {56 vars in 1114 bytes} [Mon Aug 1 15:11:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:24.109389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bd83d-a237-4818-a6cb-b6287dde0753 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:24.154183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:24.224758 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ed53d-d17e-48d2-9257-562b252dab66 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:24.236106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bd83d-a237-4818-a6cb-b6287dde0753 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:24.242672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:24.270526 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f8e7c-e7f0-4205-9467-d37b4ce0f459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:24.271943 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f8e7c-e7f0-4205-9467-d37b4ce0f459 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:24.272225 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f8e7c-e7f0-4205-9467-d37b4ce0f459 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:24.272496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f8e7c-e7f0-4205-9467-d37b4ce0f459 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:24.276667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06935b67-aa57-48a8-9e00-59268ed4b017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:24.277394 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06935b67-aa57-48a8-9e00-59268ed4b017 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:24.277742 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06935b67-aa57-48a8-9e00-59268ed4b017 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:24.278184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06935b67-aa57-48a8-9e00-59268ed4b017 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:24.278980 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06935b67-aa57-48a8-9e00-59268ed4b017 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:24.280560 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06935b67-aa57-48a8-9e00-59268ed4b017 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:24.282191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f8e7c-e7f0-4205-9467-d37b4ce0f459 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:24.283795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15f8e7c-e7f0-4205-9467-d37b4ce0f459 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:24.299480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-b15f8e7c-e7f0-4205-9467-d37b4ce0f459 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:11:24.299480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:24.299480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:24.299480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:24.299480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:24.299480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:24.299480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:24.299480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:24.299480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:24.299480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:24.299480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:24.299480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:24.299480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:11:24.299480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:11:24.299480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:11:24.299480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:24.300679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:24.301948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:24.301948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:24.301948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:11:24.301948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:11:24.301948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:11:24.301948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:11:24.301948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:11:24.301948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:11:24.301948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:11:24.301948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:24.301948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 467/924] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 15:11:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:24.307511 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 458/925] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:11:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:11:24.310004 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f8e7c-e7f0-4205-9467-d37b4ce0f459 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:24.434776 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f8e7c-e7f0-4205-9467-d37b4ce0f459 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:24.483021 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942c251-884b-4fd0-b3ec-f5648de637f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:24.483731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942c251-884b-4fd0-b3ec-f5648de637f5 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:24.484021 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942c251-884b-4fd0-b3ec-f5648de637f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:24.484293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942c251-884b-4fd0-b3ec-f5648de637f5 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:24.485150 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942c251-884b-4fd0-b3ec-f5648de637f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:24.485457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942c251-884b-4fd0-b3ec-f5648de637f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:24.487023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942c251-884b-4fd0-b3ec-f5648de637f5 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:24.507333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 468/926] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 15:11:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:24.513195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06935b67-aa57-48a8-9e00-59268ed4b017 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:24.604661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6dab6-fecf-404f-952f-c51088fe2371 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:24.604960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6dab6-fecf-404f-952f-c51088fe2371 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:24.605228 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c6dab6-fecf-404f-952f-c51088fe2371 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:24.608269 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 469/927] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:11:24.612447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a305d-5948-43b5-a0b3-a6787aa7b527 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:24.612728 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a305d-5948-43b5-a0b3-a6787aa7b527 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:24.613003 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a305d-5948-43b5-a0b3-a6787aa7b527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:24.627764 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:24.628958 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06935b67-aa57-48a8-9e00-59268ed4b017 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:24.631388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-510a305d-5948-43b5-a0b3-a6787aa7b527 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:24.632583 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-510a305d-5948-43b5-a0b3-a6787aa7b527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:24.653577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:24.668297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce748521-7af6-4e65-b3a0-b63f17ca830d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:24.669352 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce748521-7af6-4e65-b3a0-b63f17ca830d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:24.669952 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce748521-7af6-4e65-b3a0-b63f17ca830d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:24.670389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce748521-7af6-4e65-b3a0-b63f17ca830d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:24.682397 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce748521-7af6-4e65-b3a0-b63f17ca830d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:24.684211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce748521-7af6-4e65-b3a0-b63f17ca830d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:24.700477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-ce748521-7af6-4e65-b3a0-b63f17ca830d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:24.700477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:24.700477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:24.700477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:24.700477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:24.700477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:24.700477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:24.700477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:24.700477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:24.700477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:24.700477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:24.700477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:24.700477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:24.700477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:24.700477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:24.701654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:24.702874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:24.702874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:24.702874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:24.702874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:24.704844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 459/928] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:24.709073 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 470/929] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:11:24.710228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce748521-7af6-4e65-b3a0-b63f17ca830d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:24.731611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a305d-5948-43b5-a0b3-a6787aa7b527 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:24.823515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce748521-7af6-4e65-b3a0-b63f17ca830d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:24.840701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a305d-5948-43b5-a0b3-a6787aa7b527 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:24.871254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cbb3e-cca6-4dae-a6d7-aae92badf931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:24.871900 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cbb3e-cca6-4dae-a6d7-aae92badf931 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:24.872181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cbb3e-cca6-4dae-a6d7-aae92badf931 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:24.872455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cbb3e-cca6-4dae-a6d7-aae92badf931 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:24.876430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cbb3e-cca6-4dae-a6d7-aae92badf931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:24.876612 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cbb3e-cca6-4dae-a6d7-aae92badf931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:24.878180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cbb3e-cca6-4dae-a6d7-aae92badf931 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:24.885411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05500b7-de34-4edf-afe2-f5abf8784f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:24.886775 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05500b7-de34-4edf-afe2-f5abf8784f80 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:24.887229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05500b7-de34-4edf-afe2-f5abf8784f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:24.888098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05500b7-de34-4edf-afe2-f5abf8784f80 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:24.888911 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05500b7-de34-4edf-afe2-f5abf8784f80 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:24.890793 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05500b7-de34-4edf-afe2-f5abf8784f80 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:24.896682 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 460/930] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:24.897721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cbb3e-cca6-4dae-a6d7-aae92badf931 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:24.908755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-e05500b7-de34-4edf-afe2-f5abf8784f80 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:11:24.908755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:24.908755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:24.908755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:24.908755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:24.908755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:24.908755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:24.908755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:24.908755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:11:24.908755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:24.908755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:24.908755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:24.908755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:24.908755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:24.908755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:24.908755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:24.909770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:24.909770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:24.909770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:24.909770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:24.909770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:24.909770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:24.909770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:24.909770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:24.909770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:24.909770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:24.909770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:24.909770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:24.909770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:24.909770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:24.909770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:24.909770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:24.910638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:24.910638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:24.910638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:24.910638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:24.910638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:24.910638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:24.910638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:24.910638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:24.910638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:24.910638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:24.910638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:24.910638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:24.910638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:24.910638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:11:24.910638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:24.913422 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:24.913799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 471/931] 10.222.0.48 () {56 vars in 1132 bytes} [Mon Aug 1 15:11:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:24.925805 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05500b7-de34-4edf-afe2-f5abf8784f80 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:24.951941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:25.014377 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cbb3e-cca6-4dae-a6d7-aae92badf931 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:25.052172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53e120-dcc5-4281-b781-5ed4f13b6fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:25.052991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53e120-dcc5-4281-b781-5ed4f13b6fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:25.053378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53e120-dcc5-4281-b781-5ed4f13b6fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:25.053715 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53e120-dcc5-4281-b781-5ed4f13b6fa1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:25.059895 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05500b7-de34-4edf-afe2-f5abf8784f80 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:25.061541 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53e120-dcc5-4281-b781-5ed4f13b6fa1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:25.063227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53e120-dcc5-4281-b781-5ed4f13b6fa1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:25.075542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-2b53e120-dcc5-4281-b781-5ed4f13b6fa1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:25.075542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:25.075542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:25.075542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:25.075542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:25.075542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:25.075542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:25.075542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:25.075542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:25.075542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:25.075542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:25.075542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:25.075542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:11:25.075542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:11:25.075542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:25.076706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:25.077861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:11:25.077861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:11:25.077861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:11:25.077861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:11:25.077861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:11:25.077861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:11:25.077861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:25.077861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:25.079455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 461/932] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:25.084878 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53e120-dcc5-4281-b781-5ed4f13b6fa1 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:25.109987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfbd2a-f07d-4fd0-bc93-b57216f64ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:25.111245 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfbd2a-f07d-4fd0-bc93-b57216f64ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:25.111970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfbd2a-f07d-4fd0-bc93-b57216f64ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:25.112819 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bfbd2a-f07d-4fd0-bc93-b57216f64ca1 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:25.113987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bfbd2a-f07d-4fd0-bc93-b57216f64ca1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:25.114485 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bfbd2a-f07d-4fd0-bc93-b57216f64ca1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:25.117161 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bfbd2a-f07d-4fd0-bc93-b57216f64ca1 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:25.130395 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 472/933] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 15:11:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:25.132977 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfbd2a-f07d-4fd0-bc93-b57216f64ca1 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:25.154535 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:25.203279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53e120-dcc5-4281-b781-5ed4f13b6fa1 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:25.249529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32829a30-f01c-4298-8191-3fdd12cb0796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:25.250430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32829a30-f01c-4298-8191-3fdd12cb0796 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:25.250713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32829a30-f01c-4298-8191-3fdd12cb0796 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:25.250989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32829a30-f01c-4298-8191-3fdd12cb0796 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:25.251759 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32829a30-f01c-4298-8191-3fdd12cb0796 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:25.252496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:25.252604 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32829a30-f01c-4298-8191-3fdd12cb0796 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:25.255088 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32829a30-f01c-4298-8191-3fdd12cb0796 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:25.271902 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 462/934] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:11:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:25.277219 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32829a30-f01c-4298-8191-3fdd12cb0796 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:25.282658 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bfbd2a-f07d-4fd0-bc93-b57216f64ca1 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:25.318399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b04a3-60b2-467c-96f4-31258951db4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:25.319394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b04a3-60b2-467c-96f4-31258951db4a None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:25.319862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b04a3-60b2-467c-96f4-31258951db4a None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:25.320518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b04a3-60b2-467c-96f4-31258951db4a None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:25.321390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b04a3-60b2-467c-96f4-31258951db4a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:25.321932 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b04a3-60b2-467c-96f4-31258951db4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:25.323497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b04a3-60b2-467c-96f4-31258951db4a None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:25.333691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 473/935] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 15:11:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:25.335236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b04a3-60b2-467c-96f4-31258951db4a None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:25.376940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32829a30-f01c-4298-8191-3fdd12cb0796 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:25.402222 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:25.414963 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cd1ed-2184-498b-8525-ef2f79757eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:25.416033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cd1ed-2184-498b-8525-ef2f79757eaa None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:25.416539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cd1ed-2184-498b-8525-ef2f79757eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:25.416999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cd1ed-2184-498b-8525-ef2f79757eaa None admin] PATH_INFO: `/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/38b18e5d1e8843c6a08b1f5609cc60d5/roles/8d5e26281fb04bc68fc31532f2fb3a8d` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:25.419726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cd1ed-2184-498b-8525-ef2f79757eaa None admin] RBAC: Authorizing `identity:create_grant(project_id=218a9ee5cb2842248b6995d2562bb74b, user_id=38b18e5d1e8843c6a08b1f5609cc60d5, role_id=8d5e26281fb04bc68fc31532f2fb3a8d)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:25.421250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cd1ed-2184-498b-8525-ef2f79757eaa None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:25.437640 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 463/936] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:11:25 2022] PUT /identity/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/38b18e5d1e8843c6a08b1f5609cc60d5/roles/8d5e26281fb04bc68fc31532f2fb3a8d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:11:25.443799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b04a3-60b2-467c-96f4-31258951db4a None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:25.486866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124b2de-eaef-4f40-8769-a54ef2425535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:25.487916 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124b2de-eaef-4f40-8769-a54ef2425535 None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:25.488392 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124b2de-eaef-4f40-8769-a54ef2425535 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:25.488857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124b2de-eaef-4f40-8769-a54ef2425535 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:25.489799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124b2de-eaef-4f40-8769-a54ef2425535 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:25.491410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124b2de-eaef-4f40-8769-a54ef2425535 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:25.526139 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 474/937] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:11:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:11:26.136116 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a359a-5e79-4e0b-850b-f8c0125547a3 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:26.136620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a359a-5e79-4e0b-850b-f8c0125547a3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:26.136620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a359a-5e79-4e0b-850b-f8c0125547a3 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:26.138242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 464/938] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:26.143749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c9533-4863-4986-a2b9-6351506fc103 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:26.144196 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c9533-4863-4986-a2b9-6351506fc103 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:26.144376 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c9533-4863-4986-a2b9-6351506fc103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:26.152819 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:26.166905 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-578c9533-4863-4986-a2b9-6351506fc103 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:26.169145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-578c9533-4863-4986-a2b9-6351506fc103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:26.249975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 475/939] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:26.272181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a359a-5e79-4e0b-850b-f8c0125547a3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:26.392507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a359a-5e79-4e0b-850b-f8c0125547a3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:26.439124 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17baebdd-e2f8-4948-a748-40183f058f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:26.439776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17baebdd-e2f8-4948-a748-40183f058f60 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:26.440050 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17baebdd-e2f8-4948-a748-40183f058f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:26.440325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17baebdd-e2f8-4948-a748-40183f058f60 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:26.441049 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17baebdd-e2f8-4948-a748-40183f058f60 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:26.442528 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17baebdd-e2f8-4948-a748-40183f058f60 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-17baebdd-e2f8-4948-a748-40183f058f60 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:26.452493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:26.453534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:26.453534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:26.453534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:26.453534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:26.453534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:26.453534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:26.453534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:26.453534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:26.453534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:26.453534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:26.453534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:26.453534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:26.453534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:26.453534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:26.453534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:26.453534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:26.454543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:26.454543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:26.454543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:26.454543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:26.454543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:26.454543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:26.454543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:26.454543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:26.454543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:26.454543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:26.454543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:26.454543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:26.454543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:11:26.454543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:26.454543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 465/940] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 15:11:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:26.459786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c9533-4863-4986-a2b9-6351506fc103 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:26.468245 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16597c-073a-4ca2-96f8-a510abb08ac9 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:26.468584 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16597c-073a-4ca2-96f8-a510abb08ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:26.468798 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16597c-073a-4ca2-96f8-a510abb08ac9 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:26.470503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 466/941] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:11:26.475948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8f442-a782-4d2a-805e-6fe3063a9222 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:26.476229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8f442-a782-4d2a-805e-6fe3063a9222 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:26.476525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8f442-a782-4d2a-805e-6fe3063a9222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:26.496710 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-50f8f442-a782-4d2a-805e-6fe3063a9222 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:26.498153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-50f8f442-a782-4d2a-805e-6fe3063a9222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:26.553888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:26.582481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c9533-4863-4986-a2b9-6351506fc103 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:26.583740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 467/942] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:11:26.606991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8f442-a782-4d2a-805e-6fe3063a9222 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:26.619035 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:26.622806 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c74e0-6274-4c92-b0a2-fb186777ae92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:26.623549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c74e0-6274-4c92-b0a2-fb186777ae92 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:26.624213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c74e0-6274-4c92-b0a2-fb186777ae92 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:26.624638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c74e0-6274-4c92-b0a2-fb186777ae92 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:26.626085 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c74e0-6274-4c92-b0a2-fb186777ae92 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:26.626558 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c74e0-6274-4c92-b0a2-fb186777ae92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:26.628198 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c74e0-6274-4c92-b0a2-fb186777ae92 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:26.634482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 476/943] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 15:11:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:26.640744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c74e0-6274-4c92-b0a2-fb186777ae92 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:26.728096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8f442-a782-4d2a-805e-6fe3063a9222 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:26.766375 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c74e0-6274-4c92-b0a2-fb186777ae92 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:26.768656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266d6a7-d54d-411c-aa36-75efa0e94800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:26.769484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266d6a7-d54d-411c-aa36-75efa0e94800 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:26.770007 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266d6a7-d54d-411c-aa36-75efa0e94800 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:26.770472 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266d6a7-d54d-411c-aa36-75efa0e94800 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:26.771343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266d6a7-d54d-411c-aa36-75efa0e94800 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:26.773152 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266d6a7-d54d-411c-aa36-75efa0e94800 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:26.776325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: WARNING keystone.server.flask.common [None req-3266d6a7-d54d-411c-aa36-75efa0e94800 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 15:11:26.798192 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.sql.core [None req-3266d6a7-d54d-411c-aa36-75efa0e94800 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 15:11:26.798192 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 15:11:26.798192 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [parameters: {'id': '219bc915efb24adea5a7ccb62b8d6da8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 15:11:26.798192 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80291) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 15:11:26.810166 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370f937-09a8-497e-9b63-7f480686387b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:26.811124 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370f937-09a8-497e-9b63-7f480686387b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:26.811684 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370f937-09a8-497e-9b63-7f480686387b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:26.812156 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370f937-09a8-497e-9b63-7f480686387b None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:26.813127 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370f937-09a8-497e-9b63-7f480686387b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:26.813773 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370f937-09a8-497e-9b63-7f480686387b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:26.815197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-3266d6a7-d54d-411c-aa36-75efa0e94800 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 219bc915efb24adea5a7ccb62b8d6da8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 219bc915efb24adea5a7ccb62b8d6da8. Aug 01 15:11:26.815197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:26.815197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:11:26.815197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:11:26.815197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:11:26.815197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:11:26.815197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:11:26.815197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:11:26.815197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:11:26.815197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:11:26.815197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:11:26.815197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:11:26.815197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application result.read() Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 01 15:11:26.816000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application yield resource Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise self.value Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application session.commit() Aug 01 15:11:26.817036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self.session.flush() Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application compat.raise_( Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 15:11:26.818006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 15:11:26.818905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 15:11:26.818905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 15:11:26.818905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 15:11:26.818905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 15:11:26.818905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 15:11:26.818905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 15:11:26.818905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 15:11:26.818905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 15:11:26.818905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 15:11:26.818905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 15:11:26.818905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 15:11:26.818905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 15:11:26.818905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 15:11:26.818905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 15:11:26.818905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application result.read() Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:11:26.821567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:11:26.824047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:11:26.824047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:11:26.824047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:11:26.824047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:11:26.824047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:11:26.824047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:11:26.824047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 15:11:26.824047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 15:11:26.824047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [parameters: {'id': '219bc915efb24adea5a7ccb62b8d6da8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 15:11:26.824047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 15:11:26.824047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:26.824047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:26.824047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:26.824047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:26.824047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:26.826375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:26.828824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:26.828824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 01 15:11:26.828824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 01 15:11:26.828824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:26.828824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:26.828824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 01 15:11:26.828824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 01 15:11:26.828824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 219bc915efb24adea5a7ccb62b8d6da8. Aug 01 15:11:26.828824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:26.828824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 468/944] 10.222.0.48 () {60 vars in 1140 bytes} [Mon Aug 1 15:11:26 2022] POST /identity/v3/projects => generated 262 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:26.830785 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a370f937-09a8-497e-9b63-7f480686387b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:26.831880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266d6a7-d54d-411c-aa36-75efa0e94800 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:26.832054 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 477/945] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 15:11:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:26.836109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370f937-09a8-497e-9b63-7f480686387b None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:26.888087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:26.917078 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:26.961685 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370f937-09a8-497e-9b63-7f480686387b None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:26.982485 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266d6a7-d54d-411c-aa36-75efa0e94800 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:26.998436 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f28cb-48d0-4656-b381-b29fb0d6fe40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:26.999119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f28cb-48d0-4656-b381-b29fb0d6fe40 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:26.999464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f28cb-48d0-4656-b381-b29fb0d6fe40 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:26.999805 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f28cb-48d0-4656-b381-b29fb0d6fe40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:27.000637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f28cb-48d0-4656-b381-b29fb0d6fe40 None admin] RBAC: Adding query filter params (interface=public, service_id=e660e3c59bbf4442bf3c005ab43269b4, region_id=RegionOne) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:27.001018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f28cb-48d0-4656-b381-b29fb0d6fe40 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:27.002499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f28cb-48d0-4656-b381-b29fb0d6fe40 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:27.012659 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 478/946] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 15:11:26 2022] GET /identity/v3/endpoints?service_id=e660e3c59bbf4442bf3c005ab43269b4&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:27.014598 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f28cb-48d0-4656-b381-b29fb0d6fe40 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:27.034591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dacafe-737f-4262-9019-bee15dde8585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:27.035897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dacafe-737f-4262-9019-bee15dde8585 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:27.036455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dacafe-737f-4262-9019-bee15dde8585 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:27.036923 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dacafe-737f-4262-9019-bee15dde8585 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:27.046835 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dacafe-737f-4262-9019-bee15dde8585 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:27.049621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dacafe-737f-4262-9019-bee15dde8585 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:27.064309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-51dacafe-737f-4262-9019-bee15dde8585 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:27.064309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:27.064309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:27.064309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:27.064309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:27.064309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:27.064309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:27.064309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:27.064309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:27.064309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:27.064309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:27.064309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:27.064309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:11:27.064309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:11:27.064309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:27.065277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:27.066214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:11:27.066214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:11:27.066214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:11:27.066214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:11:27.066214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:11:27.066214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:11:27.066214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:27.066214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:27.070142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 469/947] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:27.071689 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dacafe-737f-4262-9019-bee15dde8585 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:27.142498 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f28cb-48d0-4656-b381-b29fb0d6fe40 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:27.174715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:27.187908 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:27.193133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115427b8-21ac-49c8-96d3-baca690b1ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:27.194464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115427b8-21ac-49c8-96d3-baca690b1ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:27.194849 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115427b8-21ac-49c8-96d3-baca690b1ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:27.195225 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115427b8-21ac-49c8-96d3-baca690b1ec9 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:27.196038 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115427b8-21ac-49c8-96d3-baca690b1ec9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:27.197601 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115427b8-21ac-49c8-96d3-baca690b1ec9 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:27.202767 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dacafe-737f-4262-9019-bee15dde8585 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:27.214869 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 479/948] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 15:11:27 2022] GET /identity/v3/services => generated 1497 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:11:27.253429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce11d54-9696-4605-bc9a-4acdbfdb4738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:27.254157 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce11d54-9696-4605-bc9a-4acdbfdb4738 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:27.254441 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce11d54-9696-4605-bc9a-4acdbfdb4738 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:27.254712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce11d54-9696-4605-bc9a-4acdbfdb4738 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:27.255463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce11d54-9696-4605-bc9a-4acdbfdb4738 None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:27.255763 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce11d54-9696-4605-bc9a-4acdbfdb4738 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:27.257238 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce11d54-9696-4605-bc9a-4acdbfdb4738 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:27.282793 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 470/949] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 15:11:27 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:27.605592 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0974c-ade8-4bdc-8047-79ad63b99ae1 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:27.606013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0974c-ade8-4bdc-8047-79ad63b99ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:27.606215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0974c-ade8-4bdc-8047-79ad63b99ae1 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:27.608177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 480/950] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:27.613731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c24dd-6346-4da0-a781-08cca774240f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:27.614058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c24dd-6346-4da0-a781-08cca774240f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:27.614342 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c24dd-6346-4da0-a781-08cca774240f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:27.633660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-892c24dd-6346-4da0-a781-08cca774240f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:27.635217 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-892c24dd-6346-4da0-a781-08cca774240f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:27.663815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c7ef3-fddf-4986-9340-dec2dd5a23d8 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:27.664135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c7ef3-fddf-4986-9340-dec2dd5a23d8 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:27.664382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c7ef3-fddf-4986-9340-dec2dd5a23d8 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:27.666136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 481/951] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:11:27.671529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abc661-b5e8-4e7a-9529-f9f62a4f595c None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:27.671815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abc661-b5e8-4e7a-9529-f9f62a4f595c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:27.672093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abc661-b5e8-4e7a-9529-f9f62a4f595c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:27.690293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-b1abc661-b5e8-4e7a-9529-f9f62a4f595c None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:27.691495 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-b1abc661-b5e8-4e7a-9529-f9f62a4f595c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:27.710659 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 471/952] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:27.735749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c24dd-6346-4da0-a781-08cca774240f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:27.769901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 482/953] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:27.786222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abc661-b5e8-4e7a-9529-f9f62a4f595c None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:27.834843 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:27.856962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c24dd-6346-4da0-a781-08cca774240f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:27.900210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec7626-34cf-479c-9e76-78a1cfc719ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:27.901312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec7626-34cf-479c-9e76-78a1cfc719ef None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:27.901679 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec7626-34cf-479c-9e76-78a1cfc719ef None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:27.902372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec7626-34cf-479c-9e76-78a1cfc719ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:27.914192 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec7626-34cf-479c-9e76-78a1cfc719ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:27.919779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-47ec7626-34cf-479c-9e76-78a1cfc719ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:11:27.919779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:27.919779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:27.919779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:27.919779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:27.919779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:27.919779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:27.919779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:27.919779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:27.919779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:27.919779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:27.919779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:27.919779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:27.919779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:27.919779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:27.919779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:27.920968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:27.920968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:27.920968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:27.920968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:27.920968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:27.920968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:27.920968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:27.920968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:27.920968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:27.920968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:27.920968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:27.920968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:27.920968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:27.920968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:27.920968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:27.920968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:27.922158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:27.922158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:27.922158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:27.922158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:27.922158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:27.922158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:27.922158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:27.922158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:27.922158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:27.922158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:27.922158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:11:27.922158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:27.924692 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abc661-b5e8-4e7a-9529-f9f62a4f595c None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:27.928267 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 472/954] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 15:11:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:27.929187 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec7626-34cf-479c-9e76-78a1cfc719ef None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:27.943595 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:27.963514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a94071-bd7f-4d73-804c-123132e69f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:27.964536 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a94071-bd7f-4d73-804c-123132e69f08 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:27.965114 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a94071-bd7f-4d73-804c-123132e69f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:27.965592 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a94071-bd7f-4d73-804c-123132e69f08 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:27.966444 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a94071-bd7f-4d73-804c-123132e69f08 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:27.968132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a94071-bd7f-4d73-804c-123132e69f08 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-11a94071-bd7f-4d73-804c-123132e69f08 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:27.977582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:27.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:27.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:27.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:27.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:27.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:27.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:27.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:27.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:27.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:27.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:27.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:27.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:27.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:27.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:27.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:27.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:27.979822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:27.979822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:27.979822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:27.979822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:27.979822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:27.979822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:27.979822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:27.979822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:27.979822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:27.979822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:27.979822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:27.979822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:27.979822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:11:27.979822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:27.983208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 483/955] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:27.989911 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a94071-bd7f-4d73-804c-123132e69f08 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:28.041593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec7626-34cf-479c-9e76-78a1cfc719ef None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:28.078836 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db109116-aff9-4400-bb1e-ba8965256452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:28.079574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db109116-aff9-4400-bb1e-ba8965256452 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:28.079849 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db109116-aff9-4400-bb1e-ba8965256452 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:28.080117 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db109116-aff9-4400-bb1e-ba8965256452 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:28.080884 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db109116-aff9-4400-bb1e-ba8965256452 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:28.081184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db109116-aff9-4400-bb1e-ba8965256452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:28.082598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db109116-aff9-4400-bb1e-ba8965256452 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:28.096199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 473/956] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 15:11:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:28.096604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a94071-bd7f-4d73-804c-123132e69f08 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:28.097069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db109116-aff9-4400-bb1e-ba8965256452 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:28.101164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:28.135965 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e011eb4-3b50-4dff-9ff2-83dcf8671011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:28.136242 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e011eb4-3b50-4dff-9ff2-83dcf8671011 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:28.136588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e011eb4-3b50-4dff-9ff2-83dcf8671011 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:28.136921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e011eb4-3b50-4dff-9ff2-83dcf8671011 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:28.137693 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e011eb4-3b50-4dff-9ff2-83dcf8671011 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:28.138100 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e011eb4-3b50-4dff-9ff2-83dcf8671011 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:28.139549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e011eb4-3b50-4dff-9ff2-83dcf8671011 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:28.145437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 484/957] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 15:11:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:28.151211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e011eb4-3b50-4dff-9ff2-83dcf8671011 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:28.206905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db109116-aff9-4400-bb1e-ba8965256452 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:28.216255 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:28.243767 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e3a77-1410-4b2d-94f5-c64e41c0de5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:28.244782 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e3a77-1410-4b2d-94f5-c64e41c0de5d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:28.245300 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e3a77-1410-4b2d-94f5-c64e41c0de5d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:28.245795 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e3a77-1410-4b2d-94f5-c64e41c0de5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:28.257454 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e3a77-1410-4b2d-94f5-c64e41c0de5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:28.259267 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e3a77-1410-4b2d-94f5-c64e41c0de5d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:28.274412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-8a0e3a77-1410-4b2d-94f5-c64e41c0de5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:28.274412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:28.274412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:28.274412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:28.274412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:28.274412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:28.274412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:28.274412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:28.274412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:28.274412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:28.274412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:28.274412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:28.274412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:28.274412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:28.274412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:28.275688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:28.276638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:28.276638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:28.276638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:28.276638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:28.277619 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e011eb4-3b50-4dff-9ff2-83dcf8671011 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:28.284116 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 474/958] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:28.285272 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e3a77-1410-4b2d-94f5-c64e41c0de5d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:28.315696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ccb9e-042e-454a-b7d8-1b7e43a370dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:28.316321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ccb9e-042e-454a-b7d8-1b7e43a370dc None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:28.316603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ccb9e-042e-454a-b7d8-1b7e43a370dc None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:28.316903 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ccb9e-042e-454a-b7d8-1b7e43a370dc None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:28.317710 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ccb9e-042e-454a-b7d8-1b7e43a370dc None admin] RBAC: Adding query filter params (type=compute) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:28.318057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ccb9e-042e-454a-b7d8-1b7e43a370dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:28.319616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ccb9e-042e-454a-b7d8-1b7e43a370dc None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:28.324866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 485/959] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 15:11:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:28.351993 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:28.405289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e3a77-1410-4b2d-94f5-c64e41c0de5d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:28.446660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b556b70-43f9-4651-b972-6207f739a557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:28.447595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b556b70-43f9-4651-b972-6207f739a557 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:28.448022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b556b70-43f9-4651-b972-6207f739a557 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:28.448431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b556b70-43f9-4651-b972-6207f739a557 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:28.449284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b556b70-43f9-4651-b972-6207f739a557 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:28.449964 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b556b70-43f9-4651-b972-6207f739a557 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:28.451583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b556b70-43f9-4651-b972-6207f739a557 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:28.466052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 475/960] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:28.478351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ccb9e-042e-454a-b7d8-1b7e43a370dc None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:28.590545 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ccb9e-042e-454a-b7d8-1b7e43a370dc None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:28.600751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:28.629512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f174d2f-6953-44b4-8717-e238f39640a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:28.630070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f174d2f-6953-44b4-8717-e238f39640a6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:28.631011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f174d2f-6953-44b4-8717-e238f39640a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:28.631011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f174d2f-6953-44b4-8717-e238f39640a6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:28.639478 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f174d2f-6953-44b4-8717-e238f39640a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:28.641012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f174d2f-6953-44b4-8717-e238f39640a6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:28.652690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-0f174d2f-6953-44b4-8717-e238f39640a6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:11:28.652690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:28.652690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:28.652690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:28.652690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:28.652690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:28.652690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:28.652690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:28.652690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:28.652690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:28.652690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:28.652690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:28.652690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:11:28.652690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:11:28.652690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:11:28.652690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:28.653629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:28.654808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:28.654808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:28.654808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:11:28.654808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:11:28.654808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:11:28.654808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:11:28.654808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:11:28.654808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:11:28.654808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:11:28.654808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:28.654808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 486/961] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 15:11:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:28.660234 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b556b70-43f9-4651-b972-6207f739a557 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:28.762796 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b556b70-43f9-4651-b972-6207f739a557 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:28.780938 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:28.802521 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc91ec7-dfb8-49d1-bc15-218fb7f60ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:28.803188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc91ec7-dfb8-49d1-bc15-218fb7f60ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:28.803463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc91ec7-dfb8-49d1-bc15-218fb7f60ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:28.803728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc91ec7-dfb8-49d1-bc15-218fb7f60ad5 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:28.804545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc91ec7-dfb8-49d1-bc15-218fb7f60ad5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:28.804843 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc91ec7-dfb8-49d1-bc15-218fb7f60ad5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:28.806317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc91ec7-dfb8-49d1-bc15-218fb7f60ad5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:28.826337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 476/962] 10.222.0.48 () {56 vars in 1149 bytes} [Mon Aug 1 15:11:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:28.832253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f174d2f-6953-44b4-8717-e238f39640a6 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:28.961521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f174d2f-6953-44b4-8717-e238f39640a6 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:28.997162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ab83b-dd92-4cd8-a7ad-5ce99deee5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:28.997818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ab83b-dd92-4cd8-a7ad-5ce99deee5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:28.998090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ab83b-dd92-4cd8-a7ad-5ce99deee5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:28.998360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ab83b-dd92-4cd8-a7ad-5ce99deee5a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:29.039673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:29.081338 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ab83b-dd92-4cd8-a7ad-5ce99deee5a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:29.082841 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ab83b-dd92-4cd8-a7ad-5ce99deee5a8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:29.184528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 487/963] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:11:28 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:11:29.192153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc91ec7-dfb8-49d1-bc15-218fb7f60ad5 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:29.303979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc91ec7-dfb8-49d1-bc15-218fb7f60ad5 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:29.339956 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e460f2-0c98-4039-ac9b-4613094f6b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:29.340651 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e460f2-0c98-4039-ac9b-4613094f6b97 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:29.340929 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e460f2-0c98-4039-ac9b-4613094f6b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:29.341196 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e460f2-0c98-4039-ac9b-4613094f6b97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:29.352330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e460f2-0c98-4039-ac9b-4613094f6b97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:29.353881 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e460f2-0c98-4039-ac9b-4613094f6b97 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:29.369653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-87e460f2-0c98-4039-ac9b-4613094f6b97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:29.369653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:29.369653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:29.369653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:29.369653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:29.369653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:29.369653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:29.369653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:29.369653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:29.369653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:29.369653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:29.369653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:29.369653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:29.369653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:29.369653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:29.370620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:29.371509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:29.371509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:29.371509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:29.371509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:29.371509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 477/964] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:29.376854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ab83b-dd92-4cd8-a7ad-5ce99deee5a8 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:29.384806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d099a1-ce1b-40c2-a71d-06e6a847ccf1 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:29.385094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d099a1-ce1b-40c2-a71d-06e6a847ccf1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:29.385375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d099a1-ce1b-40c2-a71d-06e6a847ccf1 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:29.387034 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 478/965] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:11:29.392218 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa355258-4b30-4629-804a-3c522bb0f2d5 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:29.392498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa355258-4b30-4629-804a-3c522bb0f2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:29.392768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa355258-4b30-4629-804a-3c522bb0f2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:29.411235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-fa355258-4b30-4629-804a-3c522bb0f2d5 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:29.412443 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-fa355258-4b30-4629-804a-3c522bb0f2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:29.436108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:29.451872 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:29.488667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ab83b-dd92-4cd8-a7ad-5ce99deee5a8 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:29.493973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 479/966] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:11:29.493973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75feb344-bed6-4297-8122-4020e913d40e None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:29.493973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75feb344-bed6-4297-8122-4020e913d40e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:29.493973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75feb344-bed6-4297-8122-4020e913d40e None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:29.493973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 480/967] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:29.497720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb8778-3a8f-4c54-bc67-c8f538d7bae8 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:29.498027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb8778-3a8f-4c54-bc67-c8f538d7bae8 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:29.498295 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb8778-3a8f-4c54-bc67-c8f538d7bae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:29.522597 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-97eb8778-3a8f-4c54-bc67-c8f538d7bae8 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:29.523219 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-97eb8778-3a8f-4c54-bc67-c8f538d7bae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:29.536094 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943b78c-bc72-4d29-bcd0-13cc9116bd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:29.537063 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943b78c-bc72-4d29-bcd0-13cc9116bd5e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:29.537528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943b78c-bc72-4d29-bcd0-13cc9116bd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:29.538119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943b78c-bc72-4d29-bcd0-13cc9116bd5e None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:29.539103 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943b78c-bc72-4d29-bcd0-13cc9116bd5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:29.539579 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943b78c-bc72-4d29-bcd0-13cc9116bd5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:29.541317 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943b78c-bc72-4d29-bcd0-13cc9116bd5e None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:29.556015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 488/968] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:29.557226 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943b78c-bc72-4d29-bcd0-13cc9116bd5e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:29.612685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 481/969] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:11:29.613943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb8778-3a8f-4c54-bc67-c8f538d7bae8 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:29.672715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943b78c-bc72-4d29-bcd0-13cc9116bd5e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:29.701916 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:29.707603 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:29.711790 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0198041-fcef-468b-b754-43625f5ec420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:29.712828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0198041-fcef-468b-b754-43625f5ec420 None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:29.713028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0198041-fcef-468b-b754-43625f5ec420 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:29.713287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0198041-fcef-468b-b754-43625f5ec420 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:29.714161 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0198041-fcef-468b-b754-43625f5ec420 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:29.716193 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0198041-fcef-468b-b754-43625f5ec420 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:29.719759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: WARNING keystone.server.flask.common [None req-c0198041-fcef-468b-b754-43625f5ec420 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 15:11:29.722825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb8778-3a8f-4c54-bc67-c8f538d7bae8 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:29.736457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.sql.core [None req-c0198041-fcef-468b-b754-43625f5ec420 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 15:11:29.736457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 15:11:29.736457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [parameters: {'user_id': '2ba9326c118c4b2093bb5c00bcd65af9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 15:11:29.736457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80292) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 15:11:29.746988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-c0198041-fcef-468b-b754-43625f5ec420 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 15:11:29.746988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:29.746988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:11:29.746988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:11:29.746988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:11:29.746988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:11:29.746988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:11:29.746988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:11:29.746988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:11:29.746988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:11:29.746988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:11:29.746988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:11:29.746988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:11:29.746988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application result.read() Aug 01 15:11:29.746988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:11:29.747973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application yield resource Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise self.value Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application session.commit() Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 15:11:29.749060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self.session.flush() Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application compat.raise_( Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 15:11:29.750014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 15:11:29.750924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 15:11:29.750924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 15:11:29.750924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 01 15:11:29.750924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 01 15:11:29.750924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 15:11:29.750924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 15:11:29.750924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 15:11:29.750924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 15:11:29.750924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 15:11:29.750924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 15:11:29.750924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 15:11:29.750924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 15:11:29.750924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 15:11:29.750924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 15:11:29.750924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:11:29.750924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception Aug 01 15:11:29.753006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:11:29.753006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:11:29.753006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:11:29.753006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:11:29.753006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:11:29.753006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:11:29.753006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:11:29.753006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:11:29.753006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:11:29.753006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:11:29.753006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:11:29.753006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application result.read() Aug 01 15:11:29.753006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:11:29.753006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:11:29.753006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:11:29.753006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:11:29.755992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:11:29.755992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:11:29.755992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:11:29.755992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:11:29.755992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 15:11:29.755992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 15:11:29.755992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [parameters: {'user_id': '2ba9326c118c4b2093bb5c00bcd65af9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 15:11:29.755992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 15:11:29.755992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:29.755992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:29.755992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:29.755992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:29.755992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:29.755992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:29.755992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 01 15:11:29.757823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 01 15:11:29.759031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 01 15:11:29.759031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 01 15:11:29.759031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 15:11:29.759031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 15:11:29.759031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 15:11:29.759031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:29.759031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 489/970] 10.222.0.48 () {60 vars in 1131 bytes} [Mon Aug 1 15:11:29 2022] POST /identity/v3/users => generated 160 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:29.759031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0198041-fcef-468b-b754-43625f5ec420 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:29.769581 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b5f95-cccc-4860-9c5d-4f1aa4e8059c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:29.770495 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b5f95-cccc-4860-9c5d-4f1aa4e8059c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:29.770840 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b5f95-cccc-4860-9c5d-4f1aa4e8059c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:29.771033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b5f95-cccc-4860-9c5d-4f1aa4e8059c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:29.778468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b5f95-cccc-4860-9c5d-4f1aa4e8059c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:29.780109 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b5f95-cccc-4860-9c5d-4f1aa4e8059c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:29.792591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-372b5f95-cccc-4860-9c5d-4f1aa4e8059c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:29.792591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:29.792591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:29.792591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:29.792591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:29.792591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:29.792591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:29.792591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:29.792591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:29.792591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:29.792591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:29.792591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:29.792591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:11:29.792591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:11:29.792591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:29.793692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:29.800318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:11:29.800318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:11:29.800318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:11:29.800318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:11:29.800318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:11:29.800318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:11:29.800318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:29.800318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:29.800318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 482/971] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:29.800318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b5f95-cccc-4860-9c5d-4f1aa4e8059c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:29.872729 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0198041-fcef-468b-b754-43625f5ec420 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:29.911680 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b5f95-cccc-4860-9c5d-4f1aa4e8059c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:29.913542 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854fac4-ea6c-4e71-b68c-87d7772b6920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:29.914337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854fac4-ea6c-4e71-b68c-87d7772b6920 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:29.914713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854fac4-ea6c-4e71-b68c-87d7772b6920 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:29.915089 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854fac4-ea6c-4e71-b68c-87d7772b6920 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:29.915798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854fac4-ea6c-4e71-b68c-87d7772b6920 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:29.917461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854fac4-ea6c-4e71-b68c-87d7772b6920 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-2854fac4-ea6c-4e71-b68c-87d7772b6920 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:29.927182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:29.928400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:29.928400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:29.928400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:29.928400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:29.928400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:29.928400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:29.928400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:29.928400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:29.928400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:29.928400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:29.928400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:29.928400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:29.928400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:29.928400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:29.928400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:29.928400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:29.929427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:29.929427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:29.929427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:29.929427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:29.929427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:29.929427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:29.929427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:29.929427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:29.929427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:29.929427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:29.929427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:29.929427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:29.929427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:11:29.929427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:29.933437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 490/972] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 15:11:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:29.934281 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854fac4-ea6c-4e71-b68c-87d7772b6920 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:29.951969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13a3d9-a353-45fa-8854-bf1f1f0152f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:29.952683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13a3d9-a353-45fa-8854-bf1f1f0152f5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:29.952963 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13a3d9-a353-45fa-8854-bf1f1f0152f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:29.953235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13a3d9-a353-45fa-8854-bf1f1f0152f5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:29.960718 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13a3d9-a353-45fa-8854-bf1f1f0152f5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:29.962213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13a3d9-a353-45fa-8854-bf1f1f0152f5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:29.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-4a13a3d9-a353-45fa-8854-bf1f1f0152f5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:11:29.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:29.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:29.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:29.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:29.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:29.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:29.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:29.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:29.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:29.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:29.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:29.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:11:29.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:11:29.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:11:29.974429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:29.975448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:29.976470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:29.976470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:29.976470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:11:29.976470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:11:29.976470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:11:29.976470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:11:29.976470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:11:29.976470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:11:29.976470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:11:29.976470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:29.980502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 483/973] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:11:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:29.981389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13a3d9-a353-45fa-8854-bf1f1f0152f5 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:29.991674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:29.995863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:30.049951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854fac4-ea6c-4e71-b68c-87d7772b6920 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:30.086333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3c416-5402-4d63-9923-48b879902f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:30.086966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3c416-5402-4d63-9923-48b879902f89 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:30.087249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3c416-5402-4d63-9923-48b879902f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:30.087525 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3c416-5402-4d63-9923-48b879902f89 None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:30.088296 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3c416-5402-4d63-9923-48b879902f89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:30.088604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3c416-5402-4d63-9923-48b879902f89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:30.090076 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3c416-5402-4d63-9923-48b879902f89 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:30.096371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13a3d9-a353-45fa-8854-bf1f1f0152f5 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:30.106001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 491/974] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:11:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:30.107202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3c416-5402-4d63-9923-48b879902f89 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:30.139432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490c160-5dcc-4e75-8477-2341578171a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:30.140525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490c160-5dcc-4e75-8477-2341578171a3 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:30.141072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490c160-5dcc-4e75-8477-2341578171a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:30.141530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490c160-5dcc-4e75-8477-2341578171a3 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:30.142352 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490c160-5dcc-4e75-8477-2341578171a3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:30.142656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490c160-5dcc-4e75-8477-2341578171a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:30.144149 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490c160-5dcc-4e75-8477-2341578171a3 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:30.153952 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 484/975] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 15:11:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:30.153952 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490c160-5dcc-4e75-8477-2341578171a3 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:30.222627 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3c416-5402-4d63-9923-48b879902f89 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:30.230929 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:30.238666 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:30.263829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a188e82-aefb-4457-af89-24640e0c0563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:30.263829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a188e82-aefb-4457-af89-24640e0c0563 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:30.263829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a188e82-aefb-4457-af89-24640e0c0563 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:30.264239 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a188e82-aefb-4457-af89-24640e0c0563 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:30.264905 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a188e82-aefb-4457-af89-24640e0c0563 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:30.265210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a188e82-aefb-4457-af89-24640e0c0563 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:30.266589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490c160-5dcc-4e75-8477-2341578171a3 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:30.266693 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a188e82-aefb-4457-af89-24640e0c0563 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:30.309808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 492/976] 10.222.0.48 () {56 vars in 1101 bytes} [Mon Aug 1 15:11:30 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:30.309808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a188e82-aefb-4457-af89-24640e0c0563 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:30.310104 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8104af-36ea-47f5-9b83-8c2d42df77c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:30.310104 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8104af-36ea-47f5-9b83-8c2d42df77c5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:30.310104 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8104af-36ea-47f5-9b83-8c2d42df77c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:30.310104 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8104af-36ea-47f5-9b83-8c2d42df77c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:30.310783 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8104af-36ea-47f5-9b83-8c2d42df77c5 None admin] RBAC: Adding query filter params (role.id=8d5e26281fb04bc68fc31532f2fb3a8d, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=38b18e5d1e8843c6a08b1f5609cc60d5) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:30.310783 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8104af-36ea-47f5-9b83-8c2d42df77c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:30.310783 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8104af-36ea-47f5-9b83-8c2d42df77c5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:30.322284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 485/977] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:11:30 2022] GET /identity/v3/role_assignments?user.id=38b18e5d1e8843c6a08b1f5609cc60d5&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=8d5e26281fb04bc68fc31532f2fb3a8d => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:30.434222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a188e82-aefb-4457-af89-24640e0c0563 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:30.461238 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b52ed-002e-4520-96a3-05dc0d7a0caa None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:30.461623 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b52ed-002e-4520-96a3-05dc0d7a0caa None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:30.461809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b52ed-002e-4520-96a3-05dc0d7a0caa None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:30.467576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 486/978] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:11:30.469591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70438cf-f69b-496a-896a-157adc9242e8 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:30.469909 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70438cf-f69b-496a-896a-157adc9242e8 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:30.470183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70438cf-f69b-496a-896a-157adc9242e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:30.476477 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9b1f7-5c41-4f8c-9657-bd5d260e1a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:30.477129 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9b1f7-5c41-4f8c-9657-bd5d260e1a70 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:30.477409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9b1f7-5c41-4f8c-9657-bd5d260e1a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:30.477734 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9b1f7-5c41-4f8c-9657-bd5d260e1a70 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:30.478583 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9b1f7-5c41-4f8c-9657-bd5d260e1a70 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:30.478890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9b1f7-5c41-4f8c-9657-bd5d260e1a70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:30.480345 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9b1f7-5c41-4f8c-9657-bd5d260e1a70 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:30.487584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 493/979] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 15:11:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:30.491318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a70438cf-f69b-496a-896a-157adc9242e8 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:30.492853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a70438cf-f69b-496a-896a-157adc9242e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:30.495263 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9b1f7-5c41-4f8c-9657-bd5d260e1a70 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:30.539215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:30.594432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 487/980] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:11:30.616972 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70438cf-f69b-496a-896a-157adc9242e8 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:30.622071 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9b1f7-5c41-4f8c-9657-bd5d260e1a70 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:30.666910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8047fc-437b-4f5c-9d0d-9147b410e1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:30.668002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8047fc-437b-4f5c-9d0d-9147b410e1be None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:30.668354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8047fc-437b-4f5c-9d0d-9147b410e1be None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:30.668628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8047fc-437b-4f5c-9d0d-9147b410e1be None admin] PATH_INFO: `/v3/endpoints` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:30.669397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8047fc-437b-4f5c-9d0d-9147b410e1be None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:30.670858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8047fc-437b-4f5c-9d0d-9147b410e1be None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:30.701915 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 494/981] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:11:30 2022] POST /identity/v3/endpoints => generated 349 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:11:30.709540 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:30.753736 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70438cf-f69b-496a-896a-157adc9242e8 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:30.798099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3d8d5-b910-4b3c-9536-48b0989ab4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:30.798737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3d8d5-b910-4b3c-9536-48b0989ab4a3 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:30.799011 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3d8d5-b910-4b3c-9536-48b0989ab4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:30.799277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3d8d5-b910-4b3c-9536-48b0989ab4a3 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:30.799985 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3d8d5-b910-4b3c-9536-48b0989ab4a3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:30.801430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3d8d5-b910-4b3c-9536-48b0989ab4a3 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:30.825664 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 488/982] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 15:11:30 2022] POST /identity/v3/services => generated 242 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:11:32.480248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab65a47-260d-4dbc-a3d4-65def095b18e None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:32.480801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab65a47-260d-4dbc-a3d4-65def095b18e None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:32.480801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab65a47-260d-4dbc-a3d4-65def095b18e None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:32.482718 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 495/983] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:11:32.487960 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42917be-5b9f-4d9b-bd76-6f0a699d4065 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:32.488359 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42917be-5b9f-4d9b-bd76-6f0a699d4065 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:32.488522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42917be-5b9f-4d9b-bd76-6f0a699d4065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:32.506934 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d42917be-5b9f-4d9b-bd76-6f0a699d4065 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:32.508134 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d42917be-5b9f-4d9b-bd76-6f0a699d4065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:32.589070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 489/984] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:32.612148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab65a47-260d-4dbc-a3d4-65def095b18e None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:32.734208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:32.745965 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab65a47-260d-4dbc-a3d4-65def095b18e None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:32.780957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911932c8-231e-4aec-9829-63b7e4eab599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:32.781605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911932c8-231e-4aec-9829-63b7e4eab599 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:32.781886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911932c8-231e-4aec-9829-63b7e4eab599 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:32.782175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911932c8-231e-4aec-9829-63b7e4eab599 None admin] PATH_INFO: `/v3/services/image` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:32.782801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911932c8-231e-4aec-9829-63b7e4eab599 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:32.784182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911932c8-231e-4aec-9829-63b7e4eab599 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-911932c8-231e-4aec-9829-63b7e4eab599 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:32.793549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:32.794626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:32.794626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:32.794626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:32.794626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:32.794626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:32.794626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:32.794626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:32.794626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:32.794626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:32.794626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:32.794626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:32.794626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:32.794626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:32.794626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:32.794626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:32.794626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:32.798951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:32.798951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:32.798951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:32.798951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:32.798951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:32.798951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:32.798951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:32.798951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:32.798951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:32.798951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:32.798951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:32.798951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:32.798951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:11:32.798951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:32.798951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 496/985] 10.222.0.48 () {56 vars in 1105 bytes} [Mon Aug 1 15:11:32 2022] GET /identity/v3/services/image => generated 86 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:32.803126 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42917be-5b9f-4d9b-bd76-6f0a699d4065 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:32.811462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf31d7b-683a-49e5-beca-ccacf4326594 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:32.812539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf31d7b-683a-49e5-beca-ccacf4326594 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:32.812835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf31d7b-683a-49e5-beca-ccacf4326594 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:32.814536 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 497/986] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:32.819519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a424f2-5e76-4b3e-87d1-409627482ef8 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:32.819801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a424f2-5e76-4b3e-87d1-409627482ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:32.820073 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a424f2-5e76-4b3e-87d1-409627482ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:32.839543 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-88a424f2-5e76-4b3e-87d1-409627482ef8 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:32.841132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-88a424f2-5e76-4b3e-87d1-409627482ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:32.855254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:32.917916 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 498/987] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:32.919330 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37f9fa-a735-4307-b063-7a6d3145dc72 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:32.919618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37f9fa-a735-4307-b063-7a6d3145dc72 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:32.919892 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37f9fa-a735-4307-b063-7a6d3145dc72 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:32.921255 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 499/988] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:32.926601 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310a972-0dfb-4d05-8328-fe7d71a00c81 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:32.926880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310a972-0dfb-4d05-8328-fe7d71a00c81 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:32.927149 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310a972-0dfb-4d05-8328-fe7d71a00c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:32.927629 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42917be-5b9f-4d9b-bd76-6f0a699d4065 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:32.946355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2310a972-0dfb-4d05-8328-fe7d71a00c81 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:32.947551 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2310a972-0dfb-4d05-8328-fe7d71a00c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:32.966913 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5e2c7-2559-4626-a126-5ed2863df6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:32.967551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5e2c7-2559-4626-a126-5ed2863df6bd None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:32.967830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5e2c7-2559-4626-a126-5ed2863df6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:32.968096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5e2c7-2559-4626-a126-5ed2863df6bd None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:32.968796 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5e2c7-2559-4626-a126-5ed2863df6bd None admin] RBAC: Adding query filter params (name=image) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:32.969091 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5e2c7-2559-4626-a126-5ed2863df6bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:32.970534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5e2c7-2559-4626-a126-5ed2863df6bd None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:32.975206 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 490/989] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:32.975967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5e2c7-2559-4626-a126-5ed2863df6bd None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:33.015715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 500/990] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:11:33.017004 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310a972-0dfb-4d05-8328-fe7d71a00c81 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:33.037213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:33.082133 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5e2c7-2559-4626-a126-5ed2863df6bd None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:33.093304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:33.117835 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5472c10-0ffd-4043-a644-d9d7d55b2eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:33.118715 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5472c10-0ffd-4043-a644-d9d7d55b2eab None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:33.119016 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5472c10-0ffd-4043-a644-d9d7d55b2eab None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:33.119285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5472c10-0ffd-4043-a644-d9d7d55b2eab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:33.119912 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5472c10-0ffd-4043-a644-d9d7d55b2eab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:33.121306 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5472c10-0ffd-4043-a644-d9d7d55b2eab None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:33.121614 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310a972-0dfb-4d05-8328-fe7d71a00c81 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d5472c10-0ffd-4043-a644-d9d7d55b2eab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:33.130917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:33.132144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:33.132144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:33.132144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:33.132144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:33.132144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:33.132144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:33.132144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:33.132144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:33.132144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:33.132144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:33.132144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:33.132144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:33.132144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:33.132144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:33.132144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:33.132144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:33.133020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:33.133020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:33.133020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:33.133020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:33.133020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:33.133020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:33.133020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:33.133020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:33.133020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:33.133020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:33.133020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:33.133020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:33.133020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:11:33.133020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:33.133020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 491/991] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 15:11:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:33.134035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5472c10-0ffd-4043-a644-d9d7d55b2eab None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:33.158766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8ba81-e684-4fed-a496-764768fe33dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:33.159394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8ba81-e684-4fed-a496-764768fe33dc None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:33.159673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8ba81-e684-4fed-a496-764768fe33dc None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:33.159942 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8ba81-e684-4fed-a496-764768fe33dc None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:33.160623 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8ba81-e684-4fed-a496-764768fe33dc None admin] RBAC: Adding query filter params (type=image) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:33.160928 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8ba81-e684-4fed-a496-764768fe33dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:33.162357 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e8ba81-e684-4fed-a496-764768fe33dc None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:33.166974 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 501/992] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:33 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:33.167744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8ba81-e684-4fed-a496-764768fe33dc None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:33.255365 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5472c10-0ffd-4043-a644-d9d7d55b2eab None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:33.263419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:33.292900 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8ba81-e684-4fed-a496-764768fe33dc None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:33.294902 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bb0e9-0aa5-456d-b612-492b023b9b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:33.295586 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bb0e9-0aa5-456d-b612-492b023b9b03 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:33.295936 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bb0e9-0aa5-456d-b612-492b023b9b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:33.296211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bb0e9-0aa5-456d-b612-492b023b9b03 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:33.296857 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9bb0e9-0aa5-456d-b612-492b023b9b03 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:33.298312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9bb0e9-0aa5-456d-b612-492b023b9b03 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-8f9bb0e9-0aa5-456d-b612-492b023b9b03 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:33.308107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:33.309202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:33.309202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:33.309202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:33.309202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:33.309202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:33.309202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:33.309202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:33.309202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:33.309202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:33.309202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:33.309202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:33.309202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:33.309202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:33.309202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:33.309202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:33.309202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:33.310189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:33.310189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:33.310189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:33.310189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:33.310189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:33.310189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:33.310189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:33.310189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:33.310189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:33.310189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:33.310189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:33.310189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:33.310189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:11:33.310189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:33.310189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 492/993] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:33.315812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bb0e9-0aa5-456d-b612-492b023b9b03 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:33.337977 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92b7e7-c7a8-48b0-87d7-106b11d8c0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:33.338980 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92b7e7-c7a8-48b0-87d7-106b11d8c0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:33.338980 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92b7e7-c7a8-48b0-87d7-106b11d8c0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:33.339158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92b7e7-c7a8-48b0-87d7-106b11d8c0a9 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:33.339833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92b7e7-c7a8-48b0-87d7-106b11d8c0a9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:33.340139 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92b7e7-c7a8-48b0-87d7-106b11d8c0a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:33.341524 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92b7e7-c7a8-48b0-87d7-106b11d8c0a9 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:33.347019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 502/994] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 15:11:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:33.352779 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92b7e7-c7a8-48b0-87d7-106b11d8c0a9 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:33.363429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:33.428951 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bb0e9-0aa5-456d-b612-492b023b9b03 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:33.460932 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92b7e7-c7a8-48b0-87d7-106b11d8c0a9 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:33.463714 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192248c-7c60-4cec-ab21-996bb684ceaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:33.464361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192248c-7c60-4cec-ab21-996bb684ceaf None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:33.464639 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192248c-7c60-4cec-ab21-996bb684ceaf None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:33.464911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192248c-7c60-4cec-ab21-996bb684ceaf None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:33.465601 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192248c-7c60-4cec-ab21-996bb684ceaf None admin] RBAC: Adding query filter params (name=compute) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:33.465932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192248c-7c60-4cec-ab21-996bb684ceaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:33.467322 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192248c-7c60-4cec-ab21-996bb684ceaf None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:33.472631 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 493/995] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 15:11:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:33.478127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192248c-7c60-4cec-ab21-996bb684ceaf None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:33.497334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1eaa8d-8c52-43de-baf8-5445cf1f701f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:33.498050 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1eaa8d-8c52-43de-baf8-5445cf1f701f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:33.498257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1eaa8d-8c52-43de-baf8-5445cf1f701f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:33.498531 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1eaa8d-8c52-43de-baf8-5445cf1f701f None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:33.499248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1eaa8d-8c52-43de-baf8-5445cf1f701f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:33.499552 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1eaa8d-8c52-43de-baf8-5445cf1f701f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:33.500925 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1eaa8d-8c52-43de-baf8-5445cf1f701f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:33.506696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 503/996] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 15:11:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:33.512711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1eaa8d-8c52-43de-baf8-5445cf1f701f None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:33.523738 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:33.590073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192248c-7c60-4cec-ab21-996bb684ceaf None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:33.591881 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:33.626801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1eaa8d-8c52-43de-baf8-5445cf1f701f None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:33.629146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec61ea-6c2f-40f3-9d4a-13672502adfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:33.629818 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec61ea-6c2f-40f3-9d4a-13672502adfc None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:33.630086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec61ea-6c2f-40f3-9d4a-13672502adfc None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:33.630362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec61ea-6c2f-40f3-9d4a-13672502adfc None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:33.631045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec61ea-6c2f-40f3-9d4a-13672502adfc None admin] RBAC: Adding query filter params (type=compute) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:33.631348 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec61ea-6c2f-40f3-9d4a-13672502adfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:33.632739 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec61ea-6c2f-40f3-9d4a-13672502adfc None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:33.643193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 494/997] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 15:11:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:33.644074 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec61ea-6c2f-40f3-9d4a-13672502adfc None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:33.662515 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840ba35-ba5e-4072-85d7-32595194d84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:33.663138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840ba35-ba5e-4072-85d7-32595194d84f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:33.663410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840ba35-ba5e-4072-85d7-32595194d84f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:33.663672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840ba35-ba5e-4072-85d7-32595194d84f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:33.664461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840ba35-ba5e-4072-85d7-32595194d84f None admin] RBAC: Adding query filter params (interface=admin, service_id=e660e3c59bbf4442bf3c005ab43269b4, region_id=RegionOne) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:33.664768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840ba35-ba5e-4072-85d7-32595194d84f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:33.666867 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840ba35-ba5e-4072-85d7-32595194d84f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:33.673307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 504/998] 10.222.0.48 () {56 vars in 1249 bytes} [Mon Aug 1 15:11:33 2022] GET /identity/v3/endpoints?service_id=e660e3c59bbf4442bf3c005ab43269b4&interface=admin®ion_id=RegionOne => generated 193 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:33.678796 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840ba35-ba5e-4072-85d7-32595194d84f None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:33.756994 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:33.760273 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec61ea-6c2f-40f3-9d4a-13672502adfc None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:33.795056 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840ba35-ba5e-4072-85d7-32595194d84f None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:33.804017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2ad86-2a4a-460e-b13b-919fce98d537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:33.804771 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2ad86-2a4a-460e-b13b-919fce98d537 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:33.805055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2ad86-2a4a-460e-b13b-919fce98d537 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:33.805328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2ad86-2a4a-460e-b13b-919fce98d537 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:33.806259 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2ad86-2a4a-460e-b13b-919fce98d537 None admin] RBAC: Adding query filter params (interface=public, service_id=1da733d37dac4eb3a3a703f6743308cf, region_id=RegionOne) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:33.806576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2ad86-2a4a-460e-b13b-919fce98d537 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:33.807969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2ad86-2a4a-460e-b13b-919fce98d537 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:33.814369 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 495/999] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 15:11:33 2022] GET /identity/v3/endpoints?service_id=1da733d37dac4eb3a3a703f6743308cf&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:33.819858 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2ad86-2a4a-460e-b13b-919fce98d537 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:33.830646 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad78738-503f-4add-af48-feb534043825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:33.831244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad78738-503f-4add-af48-feb534043825 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:33.831516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad78738-503f-4add-af48-feb534043825 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:33.831818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad78738-503f-4add-af48-feb534043825 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:33.832472 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad78738-503f-4add-af48-feb534043825 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:33.833837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad78738-503f-4add-af48-feb534043825 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:33.840188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 505/1000] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 15:11:33 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:11:33.853844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:33.941528 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2ad86-2a4a-460e-b13b-919fce98d537 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:33.980061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d2882-497f-4d91-a7ef-e6851d24d128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:33.980706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d2882-497f-4d91-a7ef-e6851d24d128 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:33.980985 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d2882-497f-4d91-a7ef-e6851d24d128 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:33.981258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d2882-497f-4d91-a7ef-e6851d24d128 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:33.982328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d2882-497f-4d91-a7ef-e6851d24d128 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:33.983779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d2882-497f-4d91-a7ef-e6851d24d128 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:33.991149 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 496/1001] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 15:11:33 2022] GET /identity/v3/services => generated 1727 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:11:35.173171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7b354-0a22-4986-94d3-780703440add None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:35.173672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7b354-0a22-4986-94d3-780703440add None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:35.173672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7b354-0a22-4986-94d3-780703440add None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:35.175069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 506/1002] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:35.180213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249071c-e118-4102-929c-323dfe8ae849 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:35.180599 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249071c-e118-4102-929c-323dfe8ae849 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:35.180767 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249071c-e118-4102-929c-323dfe8ae849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:35.199558 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3249071c-e118-4102-929c-323dfe8ae849 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:35.200767 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3249071c-e118-4102-929c-323dfe8ae849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:35.273680 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 497/1003] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:35.296148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7b354-0a22-4986-94d3-780703440add None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:35.399735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7b354-0a22-4986-94d3-780703440add None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:35.435000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee06a68-094f-453a-82e3-d6e3cc4620c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:35.435549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee06a68-094f-453a-82e3-d6e3cc4620c8 None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:35.435829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee06a68-094f-453a-82e3-d6e3cc4620c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:35.436101 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee06a68-094f-453a-82e3-d6e3cc4620c8 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:35.436877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee06a68-094f-453a-82e3-d6e3cc4620c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:35.438292 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee06a68-094f-453a-82e3-d6e3cc4620c8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:35.442880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:35.465810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 507/1004] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 15:11:35 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:11:35.868616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c741d7c-d00b-43e7-bfef-ffc2f3f15fdf None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:35.868616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c741d7c-d00b-43e7-bfef-ffc2f3f15fdf None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:35.868616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c741d7c-d00b-43e7-bfef-ffc2f3f15fdf None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:35.868616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 498/1005] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:35.876699 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0dd9f-3fb6-4fb8-9523-ce0488ff7cde None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:35.876699 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0dd9f-3fb6-4fb8-9523-ce0488ff7cde None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:35.876699 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0dd9f-3fb6-4fb8-9523-ce0488ff7cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:35.895164 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-8bf0dd9f-3fb6-4fb8-9523-ce0488ff7cde None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:35.896396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8bf0dd9f-3fb6-4fb8-9523-ce0488ff7cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:35.973823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 508/1006] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:35 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:35.996593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c741d7c-d00b-43e7-bfef-ffc2f3f15fdf None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:36.021545 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd885d9-426f-43a5-a838-b8525c3dc64b None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:36.021855 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd885d9-426f-43a5-a838-b8525c3dc64b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:36.022134 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd885d9-426f-43a5-a838-b8525c3dc64b None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:36.023523 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 509/1007] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:11:36.026472 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:36.028553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f648bef-ef9c-40a5-8754-43954449dfc9 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:36.028724 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f648bef-ef9c-40a5-8754-43954449dfc9 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:36.029009 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f648bef-ef9c-40a5-8754-43954449dfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:36.047404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-1f648bef-ef9c-40a5-8754-43954449dfc9 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:36.048631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1f648bef-ef9c-40a5-8754-43954449dfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:36.124239 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c741d7c-d00b-43e7-bfef-ffc2f3f15fdf None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:36.130074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 510/1008] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:36 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:11:36.152749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f648bef-ef9c-40a5-8754-43954449dfc9 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:36.162820 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72586c76-8df9-485a-b342-af8dfe625ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:36.163478 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72586c76-8df9-485a-b342-af8dfe625ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:36.163754 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72586c76-8df9-485a-b342-af8dfe625ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:36.164024 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72586c76-8df9-485a-b342-af8dfe625ea9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:36.164721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72586c76-8df9-485a-b342-af8dfe625ea9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:36.166574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72586c76-8df9-485a-b342-af8dfe625ea9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-72586c76-8df9-485a-b342-af8dfe625ea9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:36.175880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:36.176917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:36.176917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:36.176917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:36.176917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:36.176917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:36.176917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:36.176917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:36.176917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:36.176917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:36.176917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:36.176917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:36.176917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:36.176917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:36.176917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:36.176917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:36.176917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:36.177805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:36.177805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:36.177805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:36.177805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:36.177805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:36.177805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:36.177805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:36.177805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:36.177805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:36.177805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:36.177805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:36.177805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:36.177805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:11:36.177805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:36.177805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 499/1009] 10.222.0.48 () {56 vars in 1126 bytes} [Mon Aug 1 15:11:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:36.183003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72586c76-8df9-485a-b342-af8dfe625ea9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:36.213077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:36.269756 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f648bef-ef9c-40a5-8754-43954449dfc9 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:36.284644 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:36.299882 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72586c76-8df9-485a-b342-af8dfe625ea9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:36.308941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca719abc-57d8-421f-926e-276c761760e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:36.309622 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca719abc-57d8-421f-926e-276c761760e7 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:36.309927 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca719abc-57d8-421f-926e-276c761760e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:36.310198 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca719abc-57d8-421f-926e-276c761760e7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:36.310830 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca719abc-57d8-421f-926e-276c761760e7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:36.312443 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca719abc-57d8-421f-926e-276c761760e7 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-ca719abc-57d8-421f-926e-276c761760e7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:36.321822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:36.322854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:36.322854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:36.322854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:36.322854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:36.322854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:36.322854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:36.322854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:36.322854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:36.322854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:36.322854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:36.322854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:36.322854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:36.322854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:36.322854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:36.322854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:36.322854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:36.323735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:36.323735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:36.323735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:36.323735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:36.323735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:36.323735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:36.323735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:36.323735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:36.323735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:36.323735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:36.323735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:36.323735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:36.323735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:11:36.323735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:36.323735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 511/1010] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:36.328871 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca719abc-57d8-421f-926e-276c761760e7 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:36.340780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924e2f0-c667-4970-aaed-e14b02ebe1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:36.342052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924e2f0-c667-4970-aaed-e14b02ebe1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:36.342826 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924e2f0-c667-4970-aaed-e14b02ebe1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:36.343306 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924e2f0-c667-4970-aaed-e14b02ebe1d0 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:36.344357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924e2f0-c667-4970-aaed-e14b02ebe1d0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:36.345254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924e2f0-c667-4970-aaed-e14b02ebe1d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:36.347163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924e2f0-c667-4970-aaed-e14b02ebe1d0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:36.354155 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 500/1011] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 15:11:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:36.360263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924e2f0-c667-4970-aaed-e14b02ebe1d0 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:36.436739 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca719abc-57d8-421f-926e-276c761760e7 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:36.452923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:36.472951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1124a-5335-4fe6-835b-f7d7320de9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:36.473691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1124a-5335-4fe6-835b-f7d7320de9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:36.474011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1124a-5335-4fe6-835b-f7d7320de9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:36.474339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1124a-5335-4fe6-835b-f7d7320de9e8 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:36.475018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1124a-5335-4fe6-835b-f7d7320de9e8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:36.475340 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1124a-5335-4fe6-835b-f7d7320de9e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:36.476737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1124a-5335-4fe6-835b-f7d7320de9e8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:36.485807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 512/1012] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 15:11:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:36.487851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1124a-5335-4fe6-835b-f7d7320de9e8 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:36.489690 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924e2f0-c667-4970-aaed-e14b02ebe1d0 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:36.533730 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f3509-165c-4bfd-bab1-4ef5612d2eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:36.534411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f3509-165c-4bfd-bab1-4ef5612d2eff None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:36.534693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f3509-165c-4bfd-bab1-4ef5612d2eff None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:36.534970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f3509-165c-4bfd-bab1-4ef5612d2eff None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:36.535664 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f3509-165c-4bfd-bab1-4ef5612d2eff None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:36.535969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f3509-165c-4bfd-bab1-4ef5612d2eff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:36.537354 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f3509-165c-4bfd-bab1-4ef5612d2eff None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:36.543687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 501/1013] 10.222.0.48 () {56 vars in 1122 bytes} [Mon Aug 1 15:11:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:36.549536 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f3509-165c-4bfd-bab1-4ef5612d2eff None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:36.568260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:36.606931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1124a-5335-4fe6-835b-f7d7320de9e8 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:36.642941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30881c6-3b69-4ee1-a5a3-9c8bce025608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:36.643608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30881c6-3b69-4ee1-a5a3-9c8bce025608 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:36.643878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30881c6-3b69-4ee1-a5a3-9c8bce025608 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:36.644166 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30881c6-3b69-4ee1-a5a3-9c8bce025608 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:36.644848 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30881c6-3b69-4ee1-a5a3-9c8bce025608 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:36.645164 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30881c6-3b69-4ee1-a5a3-9c8bce025608 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:36.646586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30881c6-3b69-4ee1-a5a3-9c8bce025608 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:36.656467 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 513/1014] 10.222.0.48 () {56 vars in 1112 bytes} [Mon Aug 1 15:11:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:36.657454 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30881c6-3b69-4ee1-a5a3-9c8bce025608 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:36.666787 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f3509-165c-4bfd-bab1-4ef5612d2eff None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:36.700815 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f67b31-17e9-4481-a3a1-86a93eb58b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:36.701351 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67b31-17e9-4481-a3a1-86a93eb58b25 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:36.701634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67b31-17e9-4481-a3a1-86a93eb58b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:36.701994 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67b31-17e9-4481-a3a1-86a93eb58b25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:36.702707 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f67b31-17e9-4481-a3a1-86a93eb58b25 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:36.704095 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f67b31-17e9-4481-a3a1-86a93eb58b25 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:36.705158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:36.733807 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 502/1015] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:11:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:11:36.777243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30881c6-3b69-4ee1-a5a3-9c8bce025608 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:36.820613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da901d9a-d7c5-4f70-b373-fa4d91c629af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:36.821440 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da901d9a-d7c5-4f70-b373-fa4d91c629af None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:36.821727 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da901d9a-d7c5-4f70-b373-fa4d91c629af None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:36.822042 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da901d9a-d7c5-4f70-b373-fa4d91c629af None admin] PATH_INFO: `/v3/endpoints` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:36.822756 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da901d9a-d7c5-4f70-b373-fa4d91c629af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:36.824173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da901d9a-d7c5-4f70-b373-fa4d91c629af None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:36.848675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 514/1016] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:11:36 2022] POST /identity/v3/endpoints => generated 334 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:11:37.467848 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dae615-9a55-45b2-82c7-b6e6102a4342 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:37.468386 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dae615-9a55-45b2-82c7-b6e6102a4342 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:37.468386 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dae615-9a55-45b2-82c7-b6e6102a4342 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:37.469911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 503/1017] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:37.475291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8ffa1-9b73-469f-81eb-266fead8df67 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:37.475713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8ffa1-9b73-469f-81eb-266fead8df67 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:37.475809 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8ffa1-9b73-469f-81eb-266fead8df67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:37.495732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-01e8ffa1-9b73-469f-81eb-266fead8df67 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:37.496972 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-01e8ffa1-9b73-469f-81eb-266fead8df67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:37.581011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 515/1018] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:37.603944 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dae615-9a55-45b2-82c7-b6e6102a4342 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:37.689472 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:37.716361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dae615-9a55-45b2-82c7-b6e6102a4342 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:37.750417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f7372-8925-4caa-acfa-5c22e19e2b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:37.750995 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f7372-8925-4caa-acfa-5c22e19e2b81 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:37.751275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f7372-8925-4caa-acfa-5c22e19e2b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:37.751548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f7372-8925-4caa-acfa-5c22e19e2b81 None admin] PATH_INFO: `/v3/services/image` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:37.752181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f7372-8925-4caa-acfa-5c22e19e2b81 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:37.753570 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f7372-8925-4caa-acfa-5c22e19e2b81 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-162f7372-8925-4caa-acfa-5c22e19e2b81 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:37.762502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:37.763503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:37.763503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:37.763503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:37.763503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:37.763503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:37.763503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:37.763503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:37.763503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:37.763503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:37.763503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:37.763503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:37.763503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:37.763503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:37.763503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:37.763503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:37.763503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:37.769653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:37.769653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:37.769653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:37.769653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:37.769653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:37.769653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:37.769653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:37.769653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:37.769653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:37.769653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:37.769653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:37.769653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:37.769653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:11:37.769653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:37.769653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 504/1019] 10.222.0.48 () {56 vars in 1105 bytes} [Mon Aug 1 15:11:37 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:37.772056 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8ffa1-9b73-469f-81eb-266fead8df67 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:37.801617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:37.891226 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8ffa1-9b73-469f-81eb-266fead8df67 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:37.928393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fbe60-5689-4a70-8b19-3a0ff20517a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:37.929031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fbe60-5689-4a70-8b19-3a0ff20517a6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:37.929311 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fbe60-5689-4a70-8b19-3a0ff20517a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:37.929584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fbe60-5689-4a70-8b19-3a0ff20517a6 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:37.930317 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fbe60-5689-4a70-8b19-3a0ff20517a6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:37.930613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fbe60-5689-4a70-8b19-3a0ff20517a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:37.932008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fbe60-5689-4a70-8b19-3a0ff20517a6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:37.937537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 516/1020] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:37.943213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f7372-8925-4caa-acfa-5c22e19e2b81 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:38.042711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f7372-8925-4caa-acfa-5c22e19e2b81 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:38.081446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f76b6-4fb2-41ea-b76e-a169e0410360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:38.082102 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f76b6-4fb2-41ea-b76e-a169e0410360 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:38.082381 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f76b6-4fb2-41ea-b76e-a169e0410360 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:38.082651 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f76b6-4fb2-41ea-b76e-a169e0410360 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:38.083340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f76b6-4fb2-41ea-b76e-a169e0410360 None admin] RBAC: Adding query filter params (type=image) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:38.083643 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f76b6-4fb2-41ea-b76e-a169e0410360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:38.085029 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f76b6-4fb2-41ea-b76e-a169e0410360 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:38.090286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 505/1021] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:38.096072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fbe60-5689-4a70-8b19-3a0ff20517a6 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:38.185836 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:38.202273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fbe60-5689-4a70-8b19-3a0ff20517a6 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:38.237847 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a0974-a513-4b25-b6dc-1337e4601e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:38.238490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a0974-a513-4b25-b6dc-1337e4601e9c None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:38.238754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a0974-a513-4b25-b6dc-1337e4601e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:38.239033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a0974-a513-4b25-b6dc-1337e4601e9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:38.239788 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a0974-a513-4b25-b6dc-1337e4601e9c None admin] RBAC: Adding query filter params (interface=public, service_id=8505c0b5205a4f618869735697292f70, region_id=RegionOne) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:38.240099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a0974-a513-4b25-b6dc-1337e4601e9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:38.241476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a0974-a513-4b25-b6dc-1337e4601e9c None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:38.246893 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 517/1022] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 15:11:38 2022] GET /identity/v3/endpoints?service_id=8505c0b5205a4f618869735697292f70&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:38.253548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f76b6-4fb2-41ea-b76e-a169e0410360 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:38.257123 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:38.359153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f76b6-4fb2-41ea-b76e-a169e0410360 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:38.392796 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac7e01-44c1-4bc8-8fa0-4a3ef6abe470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:38.393418 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac7e01-44c1-4bc8-8fa0-4a3ef6abe470 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:38.393697 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac7e01-44c1-4bc8-8fa0-4a3ef6abe470 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:38.393995 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac7e01-44c1-4bc8-8fa0-4a3ef6abe470 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:38.394655 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac7e01-44c1-4bc8-8fa0-4a3ef6abe470 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:38.396042 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac7e01-44c1-4bc8-8fa0-4a3ef6abe470 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:38.403017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 506/1023] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 15:11:38 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:11:38.780175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608de82-90d0-4fe9-ba9c-fedd36f9cf7e None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:38.780697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608de82-90d0-4fe9-ba9c-fedd36f9cf7e None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:38.780697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608de82-90d0-4fe9-ba9c-fedd36f9cf7e None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:38.782093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 518/1024] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:38.787650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e666e-ec5d-488d-81c0-05ac249d5a60 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:38.788056 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e666e-ec5d-488d-81c0-05ac249d5a60 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:38.788226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e666e-ec5d-488d-81c0-05ac249d5a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:38.807248 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d14e666e-ec5d-488d-81c0-05ac249d5a60 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:38.808447 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d14e666e-ec5d-488d-81c0-05ac249d5a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:38.901836 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:38.910897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 507/1025] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:38.935495 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608de82-90d0-4fe9-ba9c-fedd36f9cf7e None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:38.965834 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8a85f-58ac-41cf-902f-4009dbbf4ded None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:38.966385 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8a85f-58ac-41cf-902f-4009dbbf4ded None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:38.966755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8a85f-58ac-41cf-902f-4009dbbf4ded None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:38.968439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 508/1026] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:38.973898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15b56b-c334-4a1d-8d0f-6b90606b6492 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:38.974179 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15b56b-c334-4a1d-8d0f-6b90606b6492 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:38.974450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15b56b-c334-4a1d-8d0f-6b90606b6492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:38.995589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-bb15b56b-c334-4a1d-8d0f-6b90606b6492 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:38.996807 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bb15b56b-c334-4a1d-8d0f-6b90606b6492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:39.052911 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608de82-90d0-4fe9-ba9c-fedd36f9cf7e None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:39.086237 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 509/1027] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:39.091736 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e0282-bebb-40e3-b85a-4d4871d8709a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:39.092505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e0282-bebb-40e3-b85a-4d4871d8709a None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:39.092777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e0282-bebb-40e3-b85a-4d4871d8709a None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:39.093158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e0282-bebb-40e3-b85a-4d4871d8709a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:39.094525 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e0282-bebb-40e3-b85a-4d4871d8709a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:39.095991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e0282-bebb-40e3-b85a-4d4871d8709a None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:39.102463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 519/1028] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:39.102645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15b56b-c334-4a1d-8d0f-6b90606b6492 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:39.103490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e0282-bebb-40e3-b85a-4d4871d8709a None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:39.123933 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:39.210797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15b56b-c334-4a1d-8d0f-6b90606b6492 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:39.211017 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e0282-bebb-40e3-b85a-4d4871d8709a None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:39.240252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:39.249317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4528c76-fad7-4d08-b356-f3b866789a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:39.250069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4528c76-fad7-4d08-b356-f3b866789a24 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:39.250285 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4178276-d431-4472-947a-d90cae4f6b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:39.250398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4528c76-fad7-4d08-b356-f3b866789a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:39.250610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4528c76-fad7-4d08-b356-f3b866789a24 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:39.250897 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4178276-d431-4472-947a-d90cae4f6b07 None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:39.251169 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4178276-d431-4472-947a-d90cae4f6b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:39.251434 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4178276-d431-4472-947a-d90cae4f6b07 None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:39.252110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4178276-d431-4472-947a-d90cae4f6b07 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:39.253555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4178276-d431-4472-947a-d90cae4f6b07 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:39.263839 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4528c76-fad7-4d08-b356-f3b866789a24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:39.270168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-b4528c76-fad7-4d08-b356-f3b866789a24 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:11:39.270168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:39.270168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:39.270168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:39.270168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:39.270168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:39.270168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:39.270168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:39.270168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:39.270168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:39.270168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:39.270168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:39.270168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:39.270168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:39.270168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:39.270168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:39.271164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:39.271164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:39.271164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:39.271164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:39.271164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:39.271164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:39.271164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:39.271164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:39.271164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:39.271164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:39.271164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:39.271164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:39.271164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:39.271164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:39.271164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:39.271164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:39.277534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:39.277534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:39.277534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:39.277534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:39.277534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:39.277534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:39.277534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:39.277534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:39.277534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:39.277534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:39.277534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:11:39.277534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:39.277534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 510/1029] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 15:11:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:39.277534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4528c76-fad7-4d08-b356-f3b866789a24 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:39.291320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 520/1030] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 15:11:39 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:11:39.399699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4528c76-fad7-4d08-b356-f3b866789a24 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:39.437005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f02b39-d698-48e6-859a-d04893660f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:39.437716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f02b39-d698-48e6-859a-d04893660f1f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:39.438027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f02b39-d698-48e6-859a-d04893660f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:39.438299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f02b39-d698-48e6-859a-d04893660f1f None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:39.439017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f02b39-d698-48e6-859a-d04893660f1f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:39.439350 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f02b39-d698-48e6-859a-d04893660f1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:39.440707 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f02b39-d698-48e6-859a-d04893660f1f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:39.455154 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 511/1031] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 15:11:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:39.456534 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4178276-d431-4472-947a-d90cae4f6b07 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:39.535970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:39.577614 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4178276-d431-4472-947a-d90cae4f6b07 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:39.614286 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b3090-7ebb-48de-9e0c-927c7f35ab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:39.615013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b3090-7ebb-48de-9e0c-927c7f35ab49 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:39.615299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b3090-7ebb-48de-9e0c-927c7f35ab49 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:39.615568 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b3090-7ebb-48de-9e0c-927c7f35ab49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:39.626493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b3090-7ebb-48de-9e0c-927c7f35ab49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:39.627954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b3090-7ebb-48de-9e0c-927c7f35ab49 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:39.642670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-b28b3090-7ebb-48de-9e0c-927c7f35ab49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:39.642670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:39.642670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:39.642670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:39.642670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:39.642670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:39.642670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:39.642670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:39.642670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:39.642670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:39.642670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:39.642670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:39.642670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:39.642670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:39.642670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:39.643640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:39.644575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:39.644575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:39.644575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:39.644575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:39.644575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 521/1032] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:39.650199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f02b39-d698-48e6-859a-d04893660f1f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:39.706293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:39.759369 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f02b39-d698-48e6-859a-d04893660f1f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:39.794365 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178367b-184b-4aff-801c-9b44d16818c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:39.795037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178367b-184b-4aff-801c-9b44d16818c2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:39.795295 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178367b-184b-4aff-801c-9b44d16818c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:39.797310 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178367b-184b-4aff-801c-9b44d16818c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:39.797310 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178367b-184b-4aff-801c-9b44d16818c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:39.797310 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178367b-184b-4aff-801c-9b44d16818c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:39.798034 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178367b-184b-4aff-801c-9b44d16818c2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:39.812167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 512/1033] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:39.817687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b3090-7ebb-48de-9e0c-927c7f35ab49 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:39.931481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b3090-7ebb-48de-9e0c-927c7f35ab49 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:39.955355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:39.969960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ffc00-7e64-4fbc-b10c-041b083b84ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:39.970620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ffc00-7e64-4fbc-b10c-041b083b84ee None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:39.970898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ffc00-7e64-4fbc-b10c-041b083b84ee None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:39.971171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ffc00-7e64-4fbc-b10c-041b083b84ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:39.979613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ffc00-7e64-4fbc-b10c-041b083b84ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:39.981086 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ffc00-7e64-4fbc-b10c-041b083b84ee None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:39.993576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-c79ffc00-7e64-4fbc-b10c-041b083b84ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:11:39.993576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:39.993576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:39.993576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:39.993576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:39.993576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:39.993576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:39.993576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:39.993576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:39.993576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:39.993576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:39.993576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:39.993576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:11:39.993576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:11:39.993576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:11:39.993576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:39.994576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:39.995548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:39.995548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:39.995548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:11:39.995548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:11:39.995548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:11:39.995548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:11:39.995548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:11:39.995548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:11:39.995548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:11:39.995548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:39.995548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 522/1034] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 15:11:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:40.002157 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178367b-184b-4aff-801c-9b44d16818c2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:40.108378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178367b-184b-4aff-801c-9b44d16818c2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:40.119786 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:40.145722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6096c-ae89-4f91-8a7e-f98476a50565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:40.146446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6096c-ae89-4f91-8a7e-f98476a50565 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:40.146720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6096c-ae89-4f91-8a7e-f98476a50565 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:40.146986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6096c-ae89-4f91-8a7e-f98476a50565 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:40.147788 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6096c-ae89-4f91-8a7e-f98476a50565 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:40.148087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6096c-ae89-4f91-8a7e-f98476a50565 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:40.149510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6096c-ae89-4f91-8a7e-f98476a50565 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:40.166820 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 513/1035] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 15:11:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:40.172562 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ffc00-7e64-4fbc-b10c-041b083b84ee None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:40.281550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ffc00-7e64-4fbc-b10c-041b083b84ee None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:40.316631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8badd-3a59-44e2-86ef-0da40db420eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:40.317279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8badd-3a59-44e2-86ef-0da40db420eb None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:40.317557 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8badd-3a59-44e2-86ef-0da40db420eb None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:40.317842 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8badd-3a59-44e2-86ef-0da40db420eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:40.367971 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:40.388018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8badd-3a59-44e2-86ef-0da40db420eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:40.389550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8badd-3a59-44e2-86ef-0da40db420eb None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:40.432234 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e5cc6-9b52-4d95-9ac5-619f418ba268 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:40.432603 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e5cc6-9b52-4d95-9ac5-619f418ba268 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:40.432773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e5cc6-9b52-4d95-9ac5-619f418ba268 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:40.434226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 514/1036] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:11:40.440114 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb8545-798a-4089-b53a-f16bba5fa230 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:40.440391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb8545-798a-4089-b53a-f16bba5fa230 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:40.440665 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb8545-798a-4089-b53a-f16bba5fa230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:40.460507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c0fb8545-798a-4089-b53a-f16bba5fa230 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:40.461846 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c0fb8545-798a-4089-b53a-f16bba5fa230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:40.495708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 523/1037] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:11:40 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:11:40.501405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8badd-3a59-44e2-86ef-0da40db420eb None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:40.537092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 515/1038] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:11:40.559254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb8545-798a-4089-b53a-f16bba5fa230 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:40.587007 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:40.609185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8badd-3a59-44e2-86ef-0da40db420eb None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:40.644693 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc18a4-d238-40ba-bde9-b1ef373d0206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:40.645771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc18a4-d238-40ba-bde9-b1ef373d0206 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:40.646300 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc18a4-d238-40ba-bde9-b1ef373d0206 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:40.646719 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc18a4-d238-40ba-bde9-b1ef373d0206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:40.658002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc18a4-d238-40ba-bde9-b1ef373d0206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:40.658595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:40.659855 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc18a4-d238-40ba-bde9-b1ef373d0206 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:40.662795 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb8545-798a-4089-b53a-f16bba5fa230 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:40.675954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-55fc18a4-d238-40ba-bde9-b1ef373d0206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:40.675954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:40.675954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:40.675954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:40.675954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:40.675954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:40.675954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:40.675954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:40.675954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:40.675954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:40.675954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:40.675954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:40.675954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:40.675954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:40.675954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:40.676882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:40.678519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:40.678519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:40.678519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:40.678519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:40.678519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 524/1039] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:40.682801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc18a4-d238-40ba-bde9-b1ef373d0206 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:40.701133 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f5212-3e60-455a-8682-e30e57ff5eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:40.701848 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f5212-3e60-455a-8682-e30e57ff5eca None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:40.702135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f5212-3e60-455a-8682-e30e57ff5eca None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:40.702411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f5212-3e60-455a-8682-e30e57ff5eca None admin] PATH_INFO: `/v3/services/image` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:40.703046 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f5212-3e60-455a-8682-e30e57ff5eca None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:40.704438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f5212-3e60-455a-8682-e30e57ff5eca None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-848f5212-3e60-455a-8682-e30e57ff5eca None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:40.713413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:40.714473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:40.714473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:40.714473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:40.714473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:40.714473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:40.714473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:40.714473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:11:40.714473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:11:40.714473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:11:40.714473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:11:40.714473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:40.714473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:40.714473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:40.714473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:40.714473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:40.714473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:40.715388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:40.715388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:40.715388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:40.715388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:40.715388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:40.715388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:40.715388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:40.715388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:40.715388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:40.715388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:40.715388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:11:40.715388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:11:40.715388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:11:40.715388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:40.715388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 516/1040] 10.222.0.48 () {56 vars in 1105 bytes} [Mon Aug 1 15:11:40 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:40.721462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f5212-3e60-455a-8682-e30e57ff5eca None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:40.791799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc18a4-d238-40ba-bde9-b1ef373d0206 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:40.825534 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470fdbd-ad33-4939-877a-4e43dcd3cd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:40.826210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470fdbd-ad33-4939-877a-4e43dcd3cd89 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:40.826456 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470fdbd-ad33-4939-877a-4e43dcd3cd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:40.826772 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470fdbd-ad33-4939-877a-4e43dcd3cd89 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:40.827460 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470fdbd-ad33-4939-877a-4e43dcd3cd89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:40.827764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470fdbd-ad33-4939-877a-4e43dcd3cd89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:40.829172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470fdbd-ad33-4939-877a-4e43dcd3cd89 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:40.829395 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f5212-3e60-455a-8682-e30e57ff5eca None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:40.830599 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:40.846601 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 525/1041] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:40.847368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470fdbd-ad33-4939-877a-4e43dcd3cd89 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:40.872883 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09812ab6-6663-4935-84cc-13fec0d6302e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:40.874341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09812ab6-6663-4935-84cc-13fec0d6302e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:40.874826 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09812ab6-6663-4935-84cc-13fec0d6302e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:40.875094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09812ab6-6663-4935-84cc-13fec0d6302e None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:40.875776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09812ab6-6663-4935-84cc-13fec0d6302e None admin] RBAC: Adding query filter params (name=image) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:40.876074 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09812ab6-6663-4935-84cc-13fec0d6302e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:40.877473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09812ab6-6663-4935-84cc-13fec0d6302e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:40.883337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 517/1042] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:40.888897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09812ab6-6663-4935-84cc-13fec0d6302e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:40.918281 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:40.961641 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470fdbd-ad33-4939-877a-4e43dcd3cd89 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:40.996519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f92984-e1d4-41a8-9fbc-218a331905c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:40.996899 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09812ab6-6663-4935-84cc-13fec0d6302e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:40.997167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f92984-e1d4-41a8-9fbc-218a331905c3 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:40.997437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f92984-e1d4-41a8-9fbc-218a331905c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:40.997705 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f92984-e1d4-41a8-9fbc-218a331905c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:41.004956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f92984-e1d4-41a8-9fbc-218a331905c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:41.006451 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f92984-e1d4-41a8-9fbc-218a331905c3 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:41.019236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-67f92984-e1d4-41a8-9fbc-218a331905c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:41.019236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:41.019236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:41.019236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:41.019236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:41.019236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:41.019236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:41.019236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:41.019236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:41.019236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:41.019236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:41.019236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:41.019236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:11:41.019236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:11:41.019236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:41.020186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:41.021109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:11:41.021109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:11:41.021109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:11:41.021109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:11:41.021109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:11:41.021109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:11:41.021109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:41.021109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:41.021109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 526/1043] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:41.026688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f92984-e1d4-41a8-9fbc-218a331905c3 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:41.038401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bc48a-20b6-4582-9833-c8c38d97c577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:41.039036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bc48a-20b6-4582-9833-c8c38d97c577 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:41.039505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bc48a-20b6-4582-9833-c8c38d97c577 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:41.039609 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1bc48a-20b6-4582-9833-c8c38d97c577 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:41.040396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bc48a-20b6-4582-9833-c8c38d97c577 None admin] RBAC: Adding query filter params (type=image) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:41.040760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bc48a-20b6-4582-9833-c8c38d97c577 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:41.042257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1bc48a-20b6-4582-9833-c8c38d97c577 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:41.052170 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 518/1044] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:41.053679 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bc48a-20b6-4582-9833-c8c38d97c577 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:41.088227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:41.141119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f92984-e1d4-41a8-9fbc-218a331905c3 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:41.154677 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:41.164549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1bc48a-20b6-4582-9833-c8c38d97c577 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:41.176981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d4da3-5ee7-4b1b-bc2b-853c02f8544a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:41.177606 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d4da3-5ee7-4b1b-bc2b-853c02f8544a None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:41.177913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d4da3-5ee7-4b1b-bc2b-853c02f8544a None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:41.178187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d4da3-5ee7-4b1b-bc2b-853c02f8544a None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:41.178947 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d4da3-5ee7-4b1b-bc2b-853c02f8544a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:41.179252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d4da3-5ee7-4b1b-bc2b-853c02f8544a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:41.180765 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d4da3-5ee7-4b1b-bc2b-853c02f8544a None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:41.192980 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 527/1045] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:11:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:41.200511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d4da3-5ee7-4b1b-bc2b-853c02f8544a None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:41.203269 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75587c-459e-46fc-a9c3-80107c1d2a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:41.203909 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75587c-459e-46fc-a9c3-80107c1d2a0d None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:41.204188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75587c-459e-46fc-a9c3-80107c1d2a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:41.204459 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75587c-459e-46fc-a9c3-80107c1d2a0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:41.205172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75587c-459e-46fc-a9c3-80107c1d2a0d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:41.206601 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75587c-459e-46fc-a9c3-80107c1d2a0d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:41.229522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 519/1046] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:11:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:11:41.319831 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d4da3-5ee7-4b1b-bc2b-853c02f8544a None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:41.335851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:41.347558 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f2fb7-a589-4c9b-8a70-1008c8c999c5 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:41.348003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f2fb7-a589-4c9b-8a70-1008c8c999c5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:41.348342 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f2fb7-a589-4c9b-8a70-1008c8c999c5 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:41.350124 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 520/1047] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:41.356792 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f431feb-1c80-41bd-bd5e-03d5d16c9ea0 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:41.357067 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f431feb-1c80-41bd-bd5e-03d5d16c9ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:41.357360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f431feb-1c80-41bd-bd5e-03d5d16c9ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:41.369215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986aeaed-af35-4a89-a130-332f1c72a623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:41.369895 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986aeaed-af35-4a89-a130-332f1c72a623 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:41.370172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986aeaed-af35-4a89-a130-332f1c72a623 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:41.371048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986aeaed-af35-4a89-a130-332f1c72a623 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:41.371878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986aeaed-af35-4a89-a130-332f1c72a623 None admin] RBAC: Adding query filter params (role.id=8d5e26281fb04bc68fc31532f2fb3a8d, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=2de0ff99a2b74979a28a8b10e9ebdfcd) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:41.372342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986aeaed-af35-4a89-a130-332f1c72a623 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:41.373943 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986aeaed-af35-4a89-a130-332f1c72a623 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:41.376570 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6f431feb-1c80-41bd-bd5e-03d5d16c9ea0 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:41.377804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6f431feb-1c80-41bd-bd5e-03d5d16c9ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:41.380352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 528/1048] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:11:41 2022] GET /identity/v3/role_assignments?user.id=2de0ff99a2b74979a28a8b10e9ebdfcd&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=8d5e26281fb04bc68fc31532f2fb3a8d => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:41.458591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 521/1049] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:11:41.484560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986aeaed-af35-4a89-a130-332f1c72a623 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:41.592309 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986aeaed-af35-4a89-a130-332f1c72a623 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:41.627435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452173c1-3f6c-4b23-a0f3-7c09c3b6d878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:41.628088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452173c1-3f6c-4b23-a0f3-7c09c3b6d878 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:41.628370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452173c1-3f6c-4b23-a0f3-7c09c3b6d878 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:41.628642 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452173c1-3f6c-4b23-a0f3-7c09c3b6d878 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:41.629817 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452173c1-3f6c-4b23-a0f3-7c09c3b6d878 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:41.631274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452173c1-3f6c-4b23-a0f3-7c09c3b6d878 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:41.637589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 529/1050] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:41.638848 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f431feb-1c80-41bd-bd5e-03d5d16c9ea0 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:41.715568 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:41.743801 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f431feb-1c80-41bd-bd5e-03d5d16c9ea0 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:41.779526 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094afe0-d0f2-459d-8138-7126278eac19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:41.780263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094afe0-d0f2-459d-8138-7126278eac19 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:41.780603 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094afe0-d0f2-459d-8138-7126278eac19 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:41.780934 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094afe0-d0f2-459d-8138-7126278eac19 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:41.781690 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094afe0-d0f2-459d-8138-7126278eac19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:41.783221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094afe0-d0f2-459d-8138-7126278eac19 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:41.809837 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 522/1051] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:11:41 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:11:43.344554 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44099991-65de-466d-abc3-34b287b19e2f None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:43.345106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44099991-65de-466d-abc3-34b287b19e2f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:43.345106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44099991-65de-466d-abc3-34b287b19e2f None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:43.349816 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 530/1052] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:43.352142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5872a-3599-4048-b082-5e3f8332c8b5 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:43.352596 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5872a-3599-4048-b082-5e3f8332c8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:43.352765 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5872a-3599-4048-b082-5e3f8332c8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:43.373168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-11a5872a-3599-4048-b082-5e3f8332c8b5 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:43.374469 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-11a5872a-3599-4048-b082-5e3f8332c8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:43.399055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c606f0-0a7e-4b0d-99f0-2591a32d8783 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:43.399363 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c606f0-0a7e-4b0d-99f0-2591a32d8783 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:43.399619 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c606f0-0a7e-4b0d-99f0-2591a32d8783 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:43.401172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 531/1053] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:43.406663 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4de6b-b511-48e3-a0d9-674814f84186 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:43.407088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4de6b-b511-48e3-a0d9-674814f84186 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:43.407370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4de6b-b511-48e3-a0d9-674814f84186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:43.436385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-e5a4de6b-b511-48e3-a0d9-674814f84186 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:43.437585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e5a4de6b-b511-48e3-a0d9-674814f84186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:43.457824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 523/1054] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:43.475792 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5872a-3599-4048-b082-5e3f8332c8b5 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:43.476337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:43.516937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 532/1055] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:43.539349 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4de6b-b511-48e3-a0d9-674814f84186 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:43.545594 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:43.588590 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5872a-3599-4048-b082-5e3f8332c8b5 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:43.624832 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06cb48-2204-46b3-a046-5fbd40af131b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:43.625334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06cb48-2204-46b3-a046-5fbd40af131b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:43.625616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06cb48-2204-46b3-a046-5fbd40af131b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:43.625995 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06cb48-2204-46b3-a046-5fbd40af131b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:43.647067 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4de6b-b511-48e3-a0d9-674814f84186 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:43.687239 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f23dc-8222-47fb-8ca0-4b47842999b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:43.688186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f23dc-8222-47fb-8ca0-4b47842999b1 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:43.688465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f23dc-8222-47fb-8ca0-4b47842999b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:43.688730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f23dc-8222-47fb-8ca0-4b47842999b1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:43.699577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f23dc-8222-47fb-8ca0-4b47842999b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:43.701044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06cb48-2204-46b3-a046-5fbd40af131b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:43.702597 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06cb48-2204-46b3-a046-5fbd40af131b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:43.704660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-246f23dc-8222-47fb-8ca0-4b47842999b1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:11:43.704660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:43.704660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:43.704660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:43.704660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:43.704660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:43.704660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:43.704660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:43.704660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:43.704660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:43.704660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:43.704660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:43.704660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:43.704660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:43.704660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:43.704660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:43.705604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:43.705604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:43.705604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:43.705604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:43.705604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:43.705604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:43.705604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:43.705604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:43.705604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:43.705604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:43.705604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:43.705604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:43.705604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:43.705604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:43.705604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:43.705604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:43.707544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:43.707544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:43.707544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:43.707544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:43.707544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:43.707544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:43.707544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:43.707544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:43.707544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:43.707544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:43.707544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:11:43.707544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:43.707544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 533/1056] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 15:11:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:43.712081 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f23dc-8222-47fb-8ca0-4b47842999b1 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:43.744976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:43.768377 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:43.822386 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f23dc-8222-47fb-8ca0-4b47842999b1 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:43.828035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 524/1057] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:11:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:11:43.829575 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06cb48-2204-46b3-a046-5fbd40af131b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:43.870921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c84a9a-f752-4e92-b8af-c2699523bec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:43.871577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c84a9a-f752-4e92-b8af-c2699523bec6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:43.871853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c84a9a-f752-4e92-b8af-c2699523bec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:43.872114 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c84a9a-f752-4e92-b8af-c2699523bec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:43.872837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c84a9a-f752-4e92-b8af-c2699523bec6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:43.873143 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c84a9a-f752-4e92-b8af-c2699523bec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:43.876682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c84a9a-f752-4e92-b8af-c2699523bec6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:43.884281 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 534/1058] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 15:11:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:43.890009 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c84a9a-f752-4e92-b8af-c2699523bec6 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:43.946521 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06cb48-2204-46b3-a046-5fbd40af131b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:43.981755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90829565-7dcb-4283-a0e3-4c28f01d6205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:43.982410 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90829565-7dcb-4283-a0e3-4c28f01d6205 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:43.982716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90829565-7dcb-4283-a0e3-4c28f01d6205 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:43.983052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90829565-7dcb-4283-a0e3-4c28f01d6205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:43.985567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:43.992336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c84a9a-f752-4e92-b8af-c2699523bec6 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:43.993122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90829565-7dcb-4283-a0e3-4c28f01d6205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:43.994636 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90829565-7dcb-4283-a0e3-4c28f01d6205 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:44.009608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-90829565-7dcb-4283-a0e3-4c28f01d6205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:44.009608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:44.009608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:44.009608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:44.009608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:44.009608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:44.009608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:44.009608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:44.009608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:44.009608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:44.009608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:44.009608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:44.009608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:44.009608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:44.009608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:44.010746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:44.011676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:44.011676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:44.011676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:44.011676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:44.011676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 525/1059] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:44.011981 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b697d21-89f5-4476-ac5e-64aef9f0b4d9 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:44.012252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b697d21-89f5-4476-ac5e-64aef9f0b4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:44.012526 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b697d21-89f5-4476-ac5e-64aef9f0b4d9 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:44.013743 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 526/1060] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:44.017014 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b697d21-89f5-4476-ac5e-64aef9f0b4d9 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:44.028381 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eef29d-8b78-41e2-895a-2a8735127ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:44.029033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eef29d-8b78-41e2-895a-2a8735127ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:44.029309 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eef29d-8b78-41e2-895a-2a8735127ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:44.029614 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eef29d-8b78-41e2-895a-2a8735127ed8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:44.033458 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:44.040612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eef29d-8b78-41e2-895a-2a8735127ed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:44.042035 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eef29d-8b78-41e2-895a-2a8735127ed8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:44.056561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-84eef29d-8b78-41e2-895a-2a8735127ed8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:44.056561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:44.056561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:44.056561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:44.056561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:44.056561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:44.056561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:44.056561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:44.056561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:44.056561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:44.056561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:44.056561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:44.056561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:44.056561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:44.056561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:44.057519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:44.058480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:44.058480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:44.058480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:44.058480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:44.058480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 535/1061] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:44.059342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3b5cd-0c03-433b-92e6-2afb7f0279e4 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:44.059342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3b5cd-0c03-433b-92e6-2afb7f0279e4 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:44.059503 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3b5cd-0c03-433b-92e6-2afb7f0279e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:44.076974 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2bc3b5cd-0c03-433b-92e6-2afb7f0279e4 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:44.078202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2bc3b5cd-0c03-433b-92e6-2afb7f0279e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:44.128259 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b697d21-89f5-4476-ac5e-64aef9f0b4d9 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:44.145486 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 536/1062] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:44.146370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3b5cd-0c03-433b-92e6-2afb7f0279e4 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:44.167768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85167a-4c11-4545-ad19-c4392ae8f581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:44.168386 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85167a-4c11-4545-ad19-c4392ae8f581 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:44.168661 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85167a-4c11-4545-ad19-c4392ae8f581 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:44.168929 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85167a-4c11-4545-ad19-c4392ae8f581 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:44.169656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85167a-4c11-4545-ad19-c4392ae8f581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:44.169946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85167a-4c11-4545-ad19-c4392ae8f581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:44.171412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85167a-4c11-4545-ad19-c4392ae8f581 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:44.183737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 527/1063] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:44.184549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85167a-4c11-4545-ad19-c4392ae8f581 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:44.232625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:44.260022 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3b5cd-0c03-433b-92e6-2afb7f0279e4 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:44.276432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:44.306329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcfd98-191d-4983-9653-1409b8bd75cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:44.307247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcfd98-191d-4983-9653-1409b8bd75cd None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:44.307247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcfd98-191d-4983-9653-1409b8bd75cd None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:44.307506 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcfd98-191d-4983-9653-1409b8bd75cd None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:44.308226 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dcfd98-191d-4983-9653-1409b8bd75cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:44.308569 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dcfd98-191d-4983-9653-1409b8bd75cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:44.309972 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dcfd98-191d-4983-9653-1409b8bd75cd None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:44.310792 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85167a-4c11-4545-ad19-c4392ae8f581 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:44.327591 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 537/1064] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:44.329168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcfd98-191d-4983-9653-1409b8bd75cd None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:44.358308 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4633a64-d896-4c93-93d3-e8e3e9331a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:44.358984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4633a64-d896-4c93-93d3-e8e3e9331a70 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:44.359254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4633a64-d896-4c93-93d3-e8e3e9331a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:44.359543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4633a64-d896-4c93-93d3-e8e3e9331a70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:44.375387 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4633a64-d896-4c93-93d3-e8e3e9331a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:44.381698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e4633a64-d896-4c93-93d3-e8e3e9331a70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:11:44.381698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:44.381698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:44.381698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:44.381698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:44.381698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:44.381698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:44.381698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:44.381698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:44.381698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:44.381698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:44.381698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:44.381698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:44.381698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:44.381698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:44.381698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:44.382653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:44.382653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:44.382653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:44.382653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:44.382653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:44.382653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:44.382653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:44.382653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:44.382653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:44.382653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:44.382653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:44.382653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:44.382653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:44.382653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:44.382653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:44.382653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:44.383853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:44.383853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:44.383853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:44.383853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:44.383853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:44.383853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:44.383853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:44.383853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:44.383853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:44.383853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:44.383853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:11:44.383853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:44.383853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 528/1065] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:11:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:44.389061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4633a64-d896-4c93-93d3-e8e3e9331a70 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:44.454230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcfd98-191d-4983-9653-1409b8bd75cd None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:44.491237 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c2a43-ca62-4f5b-a187-45b4ad2e9dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:44.491886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c2a43-ca62-4f5b-a187-45b4ad2e9dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:44.492161 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c2a43-ca62-4f5b-a187-45b4ad2e9dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:44.492429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c2a43-ca62-4f5b-a187-45b4ad2e9dc8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:44.495221 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:44.501340 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c2a43-ca62-4f5b-a187-45b4ad2e9dc8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:44.503002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c2a43-ca62-4f5b-a187-45b4ad2e9dc8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:44.505187 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4633a64-d896-4c93-93d3-e8e3e9331a70 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:44.514364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-b67c2a43-ca62-4f5b-a187-45b4ad2e9dc8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:11:44.514364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:44.514364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:44.514364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:44.514364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:44.514364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:44.514364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:44.514364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:44.514364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:44.514364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:44.514364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:44.514364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:44.514364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:11:44.514364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:11:44.514364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:11:44.514364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:44.515494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:44.516992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:44.516992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:44.516992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:11:44.516992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:11:44.516992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:11:44.516992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:11:44.516992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:11:44.516992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:11:44.516992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:11:44.516992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:44.516992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 538/1066] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 15:11:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:44.521324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c2a43-ca62-4f5b-a187-45b4ad2e9dc8 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:44.540118 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9317dd-6bed-4d89-96b9-eb16e47d7c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:44.540753 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9317dd-6bed-4d89-96b9-eb16e47d7c38 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:44.541060 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9317dd-6bed-4d89-96b9-eb16e47d7c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:44.541318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9317dd-6bed-4d89-96b9-eb16e47d7c38 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:44.542067 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9317dd-6bed-4d89-96b9-eb16e47d7c38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:44.542368 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9317dd-6bed-4d89-96b9-eb16e47d7c38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:44.543753 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9317dd-6bed-4d89-96b9-eb16e47d7c38 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:44.545121 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:44.553276 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 529/1067] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:11:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:44.559081 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9317dd-6bed-4d89-96b9-eb16e47d7c38 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:44.624728 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c2a43-ca62-4f5b-a187-45b4ad2e9dc8 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:44.658780 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c1fc9-1fd8-469e-9bf4-93021cde2d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:44.659320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c1fc9-1fd8-469e-9bf4-93021cde2d34 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:44.659595 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c1fc9-1fd8-469e-9bf4-93021cde2d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:44.659728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9317dd-6bed-4d89-96b9-eb16e47d7c38 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:44.659825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c1fc9-1fd8-469e-9bf4-93021cde2d34 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:44.660661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c1fc9-1fd8-469e-9bf4-93021cde2d34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:44.660940 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c1fc9-1fd8-469e-9bf4-93021cde2d34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:44.662476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c1fc9-1fd8-469e-9bf4-93021cde2d34 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:44.679858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 539/1068] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 15:11:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:44.687466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c1fc9-1fd8-469e-9bf4-93021cde2d34 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:44.700482 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14302b-0db1-4f82-9d09-dfe8a957f7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:44.701146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14302b-0db1-4f82-9d09-dfe8a957f7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:44.701421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14302b-0db1-4f82-9d09-dfe8a957f7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:44.701718 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14302b-0db1-4f82-9d09-dfe8a957f7d4 None admin] PATH_INFO: `/v3/users/f0f4cc15e3cf4ae98e5c6a6c7c86c8db` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:44.715619 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14302b-0db1-4f82-9d09-dfe8a957f7d4 None admin] RBAC: Authorizing `identity:get_user(user_id=f0f4cc15e3cf4ae98e5c6a6c7c86c8db)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:44.717159 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14302b-0db1-4f82-9d09-dfe8a957f7d4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:44.719014 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 530/1069] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:11:44 2022] GET /identity/v3/users/f0f4cc15e3cf4ae98e5c6a6c7c86c8db => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:44.724653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14302b-0db1-4f82-9d09-dfe8a957f7d4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:44.746488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:44.779109 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:44.792101 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c1fc9-1fd8-469e-9bf4-93021cde2d34 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:44.824548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14302b-0db1-4f82-9d09-dfe8a957f7d4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:44.825873 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1feaca9-89b3-4716-9215-4eafec4e39ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:44.826764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1feaca9-89b3-4716-9215-4eafec4e39ee None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:44.827039 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1feaca9-89b3-4716-9215-4eafec4e39ee None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:44.827307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1feaca9-89b3-4716-9215-4eafec4e39ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:44.837953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1feaca9-89b3-4716-9215-4eafec4e39ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:44.839422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1feaca9-89b3-4716-9215-4eafec4e39ee None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:44.854489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-c1feaca9-89b3-4716-9215-4eafec4e39ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:44.854489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:44.854489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:44.854489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:44.854489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:44.854489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:44.854489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:44.854489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:44.854489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:44.854489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:44.854489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:44.854489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:44.854489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:44.854489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:44.854489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:44.856578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:44.858461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:44.858461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:44.858461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:44.858461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:44.858461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 540/1070] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:44.859512 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a55399-4f1d-4a38-a8d5-caad52223414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:44.860147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a55399-4f1d-4a38-a8d5-caad52223414 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:44.860427 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a55399-4f1d-4a38-a8d5-caad52223414 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:44.860701 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a55399-4f1d-4a38-a8d5-caad52223414 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:44.865380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1feaca9-89b3-4716-9215-4eafec4e39ee None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:44.935548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a55399-4f1d-4a38-a8d5-caad52223414 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:44.936996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a55399-4f1d-4a38-a8d5-caad52223414 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:44.963115 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1feaca9-89b3-4716-9215-4eafec4e39ee None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:44.970272 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:44.992261 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:44.998652 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af644f6-b82a-4ec1-ad4b-b04f839b2124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:44.999284 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af644f6-b82a-4ec1-ad4b-b04f839b2124 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:44.999556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af644f6-b82a-4ec1-ad4b-b04f839b2124 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:44.999829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af644f6-b82a-4ec1-ad4b-b04f839b2124 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:45.000528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af644f6-b82a-4ec1-ad4b-b04f839b2124 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:45.000838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af644f6-b82a-4ec1-ad4b-b04f839b2124 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:45.002255 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af644f6-b82a-4ec1-ad4b-b04f839b2124 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:45.014555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 541/1071] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:45.020248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af644f6-b82a-4ec1-ad4b-b04f839b2124 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:45.041108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 531/1072] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:11:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 15:11:45.047220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a55399-4f1d-4a38-a8d5-caad52223414 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:45.119389 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af644f6-b82a-4ec1-ad4b-b04f839b2124 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:45.143953 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a55399-4f1d-4a38-a8d5-caad52223414 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:45.155068 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd979089-89ff-4790-ab49-bb9f2c95c58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:45.155497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd979089-89ff-4790-ab49-bb9f2c95c58a None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:45.155783 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd979089-89ff-4790-ab49-bb9f2c95c58a None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:45.156046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd979089-89ff-4790-ab49-bb9f2c95c58a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:45.162721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd979089-89ff-4790-ab49-bb9f2c95c58a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:45.164164 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd979089-89ff-4790-ab49-bb9f2c95c58a None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:45.175556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-fd979089-89ff-4790-ab49-bb9f2c95c58a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:45.175556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:45.175556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:45.175556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:45.175556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:45.175556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:45.175556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:45.175556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:45.175556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:45.175556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:45.175556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:45.175556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:45.175556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:11:45.175556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:11:45.175556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:45.176575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:45.177655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:11:45.177655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:11:45.177655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:11:45.177655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:11:45.177655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:11:45.177655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:11:45.177655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:45.177655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:45.177655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 542/1073] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:45.180307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095aed08-075e-41c5-be57-bfbf30a411bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:45.180948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095aed08-075e-41c5-be57-bfbf30a411bd None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:45.181224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095aed08-075e-41c5-be57-bfbf30a411bd None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:45.181497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095aed08-075e-41c5-be57-bfbf30a411bd None admin] PATH_INFO: `/v3/projects/8db3c00959fe41f28504053f31faf49c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:45.182609 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd979089-89ff-4790-ab49-bb9f2c95c58a None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:45.182957 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095aed08-075e-41c5-be57-bfbf30a411bd None admin] RBAC: Authorizing `identity:get_project(project_id=8db3c00959fe41f28504053f31faf49c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:45.184403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095aed08-075e-41c5-be57-bfbf30a411bd None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:45.186232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 532/1074] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:11:45 2022] GET /identity/v3/projects/8db3c00959fe41f28504053f31faf49c => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:45.191987 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095aed08-075e-41c5-be57-bfbf30a411bd None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:45.215169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:45.215336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:45.289866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd979089-89ff-4790-ab49-bb9f2c95c58a None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:45.292770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095aed08-075e-41c5-be57-bfbf30a411bd None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:45.323769 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dea5c8-e2a7-4ea8-a605-733fbbc09b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:45.324384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dea5c8-e2a7-4ea8-a605-733fbbc09b51 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:45.324659 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dea5c8-e2a7-4ea8-a605-733fbbc09b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:45.324927 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dea5c8-e2a7-4ea8-a605-733fbbc09b51 None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:45.325674 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dea5c8-e2a7-4ea8-a605-733fbbc09b51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:45.326014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dea5c8-e2a7-4ea8-a605-733fbbc09b51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:45.327453 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dea5c8-e2a7-4ea8-a605-733fbbc09b51 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:45.328849 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88e3a6-5124-4ed6-8153-d6cd924e3f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:45.329478 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88e3a6-5124-4ed6-8153-d6cd924e3f2e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:45.329760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88e3a6-5124-4ed6-8153-d6cd924e3f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:45.330059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88e3a6-5124-4ed6-8153-d6cd924e3f2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:45.330883 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88e3a6-5124-4ed6-8153-d6cd924e3f2e None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=8db3c00959fe41f28504053f31faf49c, user.id=f0f4cc15e3cf4ae98e5c6a6c7c86c8db) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:45.331193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88e3a6-5124-4ed6-8153-d6cd924e3f2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:45.332628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88e3a6-5124-4ed6-8153-d6cd924e3f2e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:45.338422 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 533/1075] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:11:45 2022] GET /identity/v3/role_assignments?user.id=f0f4cc15e3cf4ae98e5c6a6c7c86c8db&scope.project.id=8db3c00959fe41f28504053f31faf49c&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:45.339505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 543/1076] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:11:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:45.345715 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88e3a6-5124-4ed6-8153-d6cd924e3f2e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:45.446594 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:45.463226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88e3a6-5124-4ed6-8153-d6cd924e3f2e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:45.508666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96804f0-f82e-4692-919e-bacf7ecd38a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:45.509363 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96804f0-f82e-4692-919e-bacf7ecd38a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:45.509651 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96804f0-f82e-4692-919e-bacf7ecd38a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:45.509958 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96804f0-f82e-4692-919e-bacf7ecd38a1 None admin] PATH_INFO: `/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/2de0ff99a2b74979a28a8b10e9ebdfcd/roles/8d5e26281fb04bc68fc31532f2fb3a8d` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:45.512159 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96804f0-f82e-4692-919e-bacf7ecd38a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=218a9ee5cb2842248b6995d2562bb74b, user_id=2de0ff99a2b74979a28a8b10e9ebdfcd, role_id=8d5e26281fb04bc68fc31532f2fb3a8d)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:45.513610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96804f0-f82e-4692-919e-bacf7ecd38a1 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:45.527264 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 534/1077] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:11:45 2022] PUT /identity/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/2de0ff99a2b74979a28a8b10e9ebdfcd/roles/8d5e26281fb04bc68fc31532f2fb3a8d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:11:47.307170 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c77c7-2019-43c7-8517-4c6d1773add1 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:47.307746 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c77c7-2019-43c7-8517-4c6d1773add1 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:47.307746 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c77c7-2019-43c7-8517-4c6d1773add1 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:47.309050 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 544/1078] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:47.314522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec307095-f2b4-4c2e-bf57-373ee431d49e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:47.314933 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec307095-f2b4-4c2e-bf57-373ee431d49e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:47.315101 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec307095-f2b4-4c2e-bf57-373ee431d49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:47.334596 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ec307095-f2b4-4c2e-bf57-373ee431d49e None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:47.335823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ec307095-f2b4-4c2e-bf57-373ee431d49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:47.412848 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 535/1079] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:47.435186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c77c7-2019-43c7-8517-4c6d1773add1 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:47.501610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cfc42-4fe0-4ee0-8c35-7011667ec1c5 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:47.501901 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cfc42-4fe0-4ee0-8c35-7011667ec1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:47.502363 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137cfc42-4fe0-4ee0-8c35-7011667ec1c5 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:47.503811 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 536/1080] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:11:47.508703 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fcbaa-b468-4b51-b0ef-76437ea585c1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:47.508985 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fcbaa-b468-4b51-b0ef-76437ea585c1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:47.509252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fcbaa-b468-4b51-b0ef-76437ea585c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:47.530192 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:47.530565 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-9d3fcbaa-b468-4b51-b0ef-76437ea585c1 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:47.531835 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9d3fcbaa-b468-4b51-b0ef-76437ea585c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:47.547492 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c77c7-2019-43c7-8517-4c6d1773add1 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:47.584611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3ad3d-c71c-4023-bd7c-10b724e6647e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:47.585160 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3ad3d-c71c-4023-bd7c-10b724e6647e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:47.585430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3ad3d-c71c-4023-bd7c-10b724e6647e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:47.585695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3ad3d-c71c-4023-bd7c-10b724e6647e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:47.596325 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3ad3d-c71c-4023-bd7c-10b724e6647e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:47.601373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-b5e3ad3d-c71c-4023-bd7c-10b724e6647e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:11:47.601373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:47.601373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:47.601373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:47.601373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:47.601373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:47.601373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:47.601373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:47.601373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:47.601373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:47.601373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:47.601373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:47.601373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:47.601373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:47.601373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:47.601373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:47.602348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:47.602348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:47.602348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:47.602348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:47.602348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:47.602348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:47.602348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:47.602348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:47.602348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:47.602348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:47.602348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:47.602348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:47.602348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:47.602348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:47.602348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:47.602348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:47.603222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:47.603222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:47.603222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:47.603222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:47.603222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:47.603222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:47.603222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:47.603222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:47.603222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:47.603222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:47.603222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:11:47.603222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:47.603222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 545/1081] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:11:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:47.604833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 537/1082] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:47.608944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3ad3d-c71c-4023-bd7c-10b724e6647e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:47.627737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fcbaa-b468-4b51-b0ef-76437ea585c1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:47.635698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:47.715144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3ad3d-c71c-4023-bd7c-10b724e6647e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:47.736219 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fcbaa-b468-4b51-b0ef-76437ea585c1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:47.751090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046e23b-abe7-466c-b938-15f82de4783c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:47.751805 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046e23b-abe7-466c-b938-15f82de4783c None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:47.752085 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046e23b-abe7-466c-b938-15f82de4783c None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:47.752354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046e23b-abe7-466c-b938-15f82de4783c None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:47.753083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046e23b-abe7-466c-b938-15f82de4783c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:47.753389 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046e23b-abe7-466c-b938-15f82de4783c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:47.754864 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046e23b-abe7-466c-b938-15f82de4783c None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:47.763685 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 546/1083] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:11:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:47.769681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046e23b-abe7-466c-b938-15f82de4783c None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:47.770121 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66be817-5859-4120-96d8-c5f8b2fd3a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:47.770768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be817-5859-4120-96d8-c5f8b2fd3a26 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:47.771041 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be817-5859-4120-96d8-c5f8b2fd3a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:47.771311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be817-5859-4120-96d8-c5f8b2fd3a26 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:47.782015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:47.782273 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66be817-5859-4120-96d8-c5f8b2fd3a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:47.787667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d66be817-5859-4120-96d8-c5f8b2fd3a26 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:11:47.787667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:47.787667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:47.787667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:47.787667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:47.787667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:47.787667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:47.787667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:47.787667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:47.787667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:47.787667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:47.787667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:47.787667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:47.787667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:47.787667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:47.787667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:47.788638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:47.788638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:47.788638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:47.788638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:47.788638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:47.788638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:47.788638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:47.788638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:47.788638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:47.788638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:47.788638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:47.788638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:47.788638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:47.788638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:47.788638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:47.788638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:47.789496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:47.789496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:47.789496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:47.789496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:47.789496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:47.789496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:47.789496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:47.789496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:47.789496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:47.789496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:47.789496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:11:47.789496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:47.789496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 538/1084] 10.222.0.48 () {56 vars in 1120 bytes} [Mon Aug 1 15:11:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:47.795113 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66be817-5859-4120-96d8-c5f8b2fd3a26 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:47.870399 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:47.883558 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046e23b-abe7-466c-b938-15f82de4783c None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:47.908662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66be817-5859-4120-96d8-c5f8b2fd3a26 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:47.920576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e4a80-1b2d-4842-b383-bd527e38a149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:47.921290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e4a80-1b2d-4842-b383-bd527e38a149 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:47.921562 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e4a80-1b2d-4842-b383-bd527e38a149 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:47.921875 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e4a80-1b2d-4842-b383-bd527e38a149 None admin] PATH_INFO: `/v3/users/f0f4cc15e3cf4ae98e5c6a6c7c86c8db` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:47.923096 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e4a80-1b2d-4842-b383-bd527e38a149 None admin] RBAC: Authorizing `identity:get_user(user_id=f0f4cc15e3cf4ae98e5c6a6c7c86c8db)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:47.924602 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e4a80-1b2d-4842-b383-bd527e38a149 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:47.926399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 547/1085] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:11:47 2022] GET /identity/v3/users/f0f4cc15e3cf4ae98e5c6a6c7c86c8db => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:47.932812 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e4a80-1b2d-4842-b383-bd527e38a149 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:47.943862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c9451-c4b4-4089-8f25-d1c60f8cd0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:47.944491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9451-c4b4-4089-8f25-d1c60f8cd0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:47.944771 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9451-c4b4-4089-8f25-d1c60f8cd0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:47.945044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9451-c4b4-4089-8f25-d1c60f8cd0a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:47.946002 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c9451-c4b4-4089-8f25-d1c60f8cd0a6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:47.946115 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c9451-c4b4-4089-8f25-d1c60f8cd0a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:47.947514 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c9451-c4b4-4089-8f25-d1c60f8cd0a6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:47.955581 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 539/1086] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 15:11:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:47.961159 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c9451-c4b4-4089-8f25-d1c60f8cd0a6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:48.010543 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:48.034683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e4a80-1b2d-4842-b383-bd527e38a149 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:48.065798 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c9451-c4b4-4089-8f25-d1c60f8cd0a6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:48.072002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bfcdd-5146-4913-b805-d0ddb118bbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:48.072726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bfcdd-5146-4913-b805-d0ddb118bbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:48.073168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bfcdd-5146-4913-b805-d0ddb118bbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:48.073532 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bfcdd-5146-4913-b805-d0ddb118bbc2 None admin] PATH_INFO: `/v3/projects/8db3c00959fe41f28504053f31faf49c` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:48.074938 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244bfcdd-5146-4913-b805-d0ddb118bbc2 None admin] RBAC: Authorizing `identity:get_project(project_id=8db3c00959fe41f28504053f31faf49c)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:48.076770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244bfcdd-5146-4913-b805-d0ddb118bbc2 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:48.079020 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 548/1087] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:11:47 2022] GET /identity/v3/projects/8db3c00959fe41f28504053f31faf49c => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:48.085232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bfcdd-5146-4913-b805-d0ddb118bbc2 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:48.093108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:48.101967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2030fa-cb30-44f9-982a-63c82caa91ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:48.102917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2030fa-cb30-44f9-982a-63c82caa91ba None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:48.103193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2030fa-cb30-44f9-982a-63c82caa91ba None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:48.103474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2030fa-cb30-44f9-982a-63c82caa91ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:48.117254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2030fa-cb30-44f9-982a-63c82caa91ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:48.118908 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2030fa-cb30-44f9-982a-63c82caa91ba None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:48.134404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-7d2030fa-cb30-44f9-982a-63c82caa91ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:48.134404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:48.134404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:48.134404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:48.134404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:48.134404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:48.134404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:48.134404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:48.134404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:48.134404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:48.134404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:48.134404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:48.134404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:48.134404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:48.134404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:48.135404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:48.136316 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:48.136316 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:48.136316 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:48.136316 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:48.136316 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 540/1088] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:11:48.142036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2030fa-cb30-44f9-982a-63c82caa91ba None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:48.197818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bfcdd-5146-4913-b805-d0ddb118bbc2 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:48.238009 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254167e-5027-4117-9a15-878eafb1a0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:48.239180 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254167e-5027-4117-9a15-878eafb1a0dd None admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:48.239463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254167e-5027-4117-9a15-878eafb1a0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:48.239737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254167e-5027-4117-9a15-878eafb1a0dd None admin] PATH_INFO: `/v3/projects/8db3c00959fe41f28504053f31faf49c/users/f0f4cc15e3cf4ae98e5c6a6c7c86c8db/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:48.241846 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c254167e-5027-4117-9a15-878eafb1a0dd None admin] RBAC: Authorizing `identity:create_grant(project_id=8db3c00959fe41f28504053f31faf49c, user_id=f0f4cc15e3cf4ae98e5c6a6c7c86c8db, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:48.243331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c254167e-5027-4117-9a15-878eafb1a0dd None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:48.252556 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2030fa-cb30-44f9-982a-63c82caa91ba None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:48.258499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 549/1089] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:11:48 2022] PUT /identity/v3/projects/8db3c00959fe41f28504053f31faf49c/users/f0f4cc15e3cf4ae98e5c6a6c7c86c8db/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:11:48.295831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e44ea-cb7c-4e50-8c9d-eb52f18269fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:48.296493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e44ea-cb7c-4e50-8c9d-eb52f18269fe None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:48.296769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e44ea-cb7c-4e50-8c9d-eb52f18269fe None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:48.297043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e44ea-cb7c-4e50-8c9d-eb52f18269fe None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:48.297764 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e44ea-cb7c-4e50-8c9d-eb52f18269fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:48.298101 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e44ea-cb7c-4e50-8c9d-eb52f18269fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:48.299563 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e44ea-cb7c-4e50-8c9d-eb52f18269fe None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:48.314589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 541/1090] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:48.320288 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254167e-5027-4117-9a15-878eafb1a0dd None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:48.325460 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:48.441916 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254167e-5027-4117-9a15-878eafb1a0dd None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:48.477402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d7b9e-bd3a-4573-bda7-7fc7aff9c1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:48.478083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d7b9e-bd3a-4573-bda7-7fc7aff9c1ca None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:48.478359 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d7b9e-bd3a-4573-bda7-7fc7aff9c1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:48.478628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d7b9e-bd3a-4573-bda7-7fc7aff9c1ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:48.487226 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d7b9e-bd3a-4573-bda7-7fc7aff9c1ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:48.488677 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d7b9e-bd3a-4573-bda7-7fc7aff9c1ca None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:48.500200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-744d7b9e-bd3a-4573-bda7-7fc7aff9c1ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:11:48.500200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:48.500200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:48.500200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:48.500200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:48.500200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:48.500200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:48.500200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:48.500200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:48.500200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:48.500200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:48.500200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:48.500200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:11:48.500200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:11:48.500200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:11:48.500200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:48.501241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:48.502786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:48.502786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:48.502786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:11:48.502786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:11:48.502786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:11:48.502786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:11:48.502786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:11:48.502786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:11:48.502786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:11:48.502786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:48.502786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 550/1091] 10.222.0.48 () {56 vars in 1093 bytes} [Mon Aug 1 15:11:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:48.507446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e44ea-cb7c-4e50-8c9d-eb52f18269fe None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:48.553962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:48.614699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e44ea-cb7c-4e50-8c9d-eb52f18269fe None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:48.651856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df1759-4194-441d-945c-de285033c2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:48.651856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df1759-4194-441d-945c-de285033c2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:48.651856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df1759-4194-441d-945c-de285033c2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:48.651856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df1759-4194-441d-945c-de285033c2d2 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:48.652580 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df1759-4194-441d-945c-de285033c2d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:48.652876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df1759-4194-441d-945c-de285033c2d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:48.654362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df1759-4194-441d-945c-de285033c2d2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:48.672602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 542/1092] 10.222.0.48 () {56 vars in 1133 bytes} [Mon Aug 1 15:11:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:48.678422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d7b9e-bd3a-4573-bda7-7fc7aff9c1ca None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:48.743864 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:48.784015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d7b9e-bd3a-4573-bda7-7fc7aff9c1ca None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:48.820401 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94d99b-d3eb-47f6-bff6-f56b884bf0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:48.821276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94d99b-d3eb-47f6-bff6-f56b884bf0dc None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:48.821455 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94d99b-d3eb-47f6-bff6-f56b884bf0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:48.821738 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94d99b-d3eb-47f6-bff6-f56b884bf0dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:48.897026 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94d99b-d3eb-47f6-bff6-f56b884bf0dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:48.898561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94d99b-d3eb-47f6-bff6-f56b884bf0dc None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:48.964855 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:49.008074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 551/1093] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:11:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:11:49.014436 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df1759-4194-441d-945c-de285033c2d2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:49.112607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:49.117438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df1759-4194-441d-945c-de285033c2d2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:49.154963 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f07eb-9225-48bb-b6ea-201fb85b1d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:49.155594 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f07eb-9225-48bb-b6ea-201fb85b1d4e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:49.155872 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f07eb-9225-48bb-b6ea-201fb85b1d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:49.156140 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f07eb-9225-48bb-b6ea-201fb85b1d4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:49.166327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f07eb-9225-48bb-b6ea-201fb85b1d4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:49.167796 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f07eb-9225-48bb-b6ea-201fb85b1d4e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:49.182845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-841f07eb-9225-48bb-b6ea-201fb85b1d4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:49.182845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:49.182845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:49.182845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:49.182845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:49.182845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:49.182845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:49.182845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:49.182845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:49.182845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:49.182845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:49.182845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:49.182845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:11:49.182845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:11:49.182845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:49.183755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:49.184772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:11:49.184772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:11:49.184772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:11:49.184772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:49.184772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 543/1094] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:49.190292 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94d99b-d3eb-47f6-bff6-f56b884bf0dc None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:49.293823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94d99b-d3eb-47f6-bff6-f56b884bf0dc None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:49.330741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057468bd-5dca-4865-acdc-cbc02a4d809a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:49.331403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057468bd-5dca-4865-acdc-cbc02a4d809a None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:49.331639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057468bd-5dca-4865-acdc-cbc02a4d809a None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:49.331908 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057468bd-5dca-4865-acdc-cbc02a4d809a None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:49.332609 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057468bd-5dca-4865-acdc-cbc02a4d809a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:49.332905 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057468bd-5dca-4865-acdc-cbc02a4d809a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:49.334333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057468bd-5dca-4865-acdc-cbc02a4d809a None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:49.346406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 552/1095] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:49.352154 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f07eb-9225-48bb-b6ea-201fb85b1d4e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:49.457886 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f07eb-9225-48bb-b6ea-201fb85b1d4e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:49.493608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac0151-5587-4e12-adca-1f6b6b626b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:49.494300 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac0151-5587-4e12-adca-1f6b6b626b47 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:49.494546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac0151-5587-4e12-adca-1f6b6b626b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:49.494839 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac0151-5587-4e12-adca-1f6b6b626b47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:49.501614 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac0151-5587-4e12-adca-1f6b6b626b47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:49.503072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac0151-5587-4e12-adca-1f6b6b626b47 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:49.514492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-fbac0151-5587-4e12-adca-1f6b6b626b47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:49.514492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:49.514492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:49.514492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:49.514492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:49.514492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:49.514492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:49.514492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:49.514492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:49.514492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:49.514492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:49.514492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:49.514492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:11:49.514492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:11:49.514492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:49.515773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:49.517321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:11:49.517321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:11:49.517321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:11:49.517321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:11:49.517321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:11:49.517321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:11:49.517321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:11:49.517321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:11:49.517321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 544/1096] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:11:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:49.521699 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057468bd-5dca-4865-acdc-cbc02a4d809a None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:49.549437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:49.622586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057468bd-5dca-4865-acdc-cbc02a4d809a None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:49.658343 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d43744-aef3-40de-aef1-84e37a690747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:49.658964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d43744-aef3-40de-aef1-84e37a690747 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:49.659245 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d43744-aef3-40de-aef1-84e37a690747 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:49.659513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d43744-aef3-40de-aef1-84e37a690747 None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:49.660269 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d43744-aef3-40de-aef1-84e37a690747 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:49.660571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d43744-aef3-40de-aef1-84e37a690747 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:49.662032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d43744-aef3-40de-aef1-84e37a690747 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:49.674685 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 553/1097] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:11:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:11:49.680754 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac0151-5587-4e12-adca-1f6b6b626b47 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:49.694550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:49.787216 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac0151-5587-4e12-adca-1f6b6b626b47 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:49.822349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6d371-a003-49e0-ad0f-9980b1032da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:49.822880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6d371-a003-49e0-ad0f-9980b1032da5 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:49.823155 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6d371-a003-49e0-ad0f-9980b1032da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:49.823426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6d371-a003-49e0-ad0f-9980b1032da5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:49.824203 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6d371-a003-49e0-ad0f-9980b1032da5 None admin] RBAC: Adding query filter params (role.id=8d5e26281fb04bc68fc31532f2fb3a8d, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=2de0ff99a2b74979a28a8b10e9ebdfcd) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:49.824508 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6d371-a003-49e0-ad0f-9980b1032da5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:49.825978 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6d371-a003-49e0-ad0f-9980b1032da5 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:49.832200 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 545/1098] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:11:49 2022] GET /identity/v3/role_assignments?user.id=2de0ff99a2b74979a28a8b10e9ebdfcd&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=8d5e26281fb04bc68fc31532f2fb3a8d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:50.253934 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22db19b-8a57-47d2-9a62-eb9bff8c6251 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:50.254131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22db19b-8a57-47d2-9a62-eb9bff8c6251 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:50.254399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22db19b-8a57-47d2-9a62-eb9bff8c6251 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:50.255812 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 554/1099] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:50.261599 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833d1bd-9c1a-41b9-8d1a-d21f99820d7f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:50.261957 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833d1bd-9c1a-41b9-8d1a-d21f99820d7f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:50.262207 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833d1bd-9c1a-41b9-8d1a-d21f99820d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:50.284663 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4833d1bd-9c1a-41b9-8d1a-d21f99820d7f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:50.285918 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4833d1bd-9c1a-41b9-8d1a-d21f99820d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:50.365430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 546/1100] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:50.388115 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22db19b-8a57-47d2-9a62-eb9bff8c6251 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:50.476448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:50.502248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22db19b-8a57-47d2-9a62-eb9bff8c6251 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:50.539413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37dc24-8700-4c5d-adb6-a4d42850016b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:50.540327 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37dc24-8700-4c5d-adb6-a4d42850016b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:50.540603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37dc24-8700-4c5d-adb6-a4d42850016b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:50.540869 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37dc24-8700-4c5d-adb6-a4d42850016b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:50.554261 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37dc24-8700-4c5d-adb6-a4d42850016b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:11:50.559335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-6a37dc24-8700-4c5d-adb6-a4d42850016b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:11:50.559335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:11:50.559335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:11:50.559335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:11:50.559335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:11:50.559335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:11:50.559335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:11:50.559335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:11:50.559335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:11:50.559335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:11:50.559335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:11:50.559335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:11:50.559335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:11:50.559335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:11:50.559335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:11:50.559335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:11:50.560537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:11:50.560537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:11:50.560537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:11:50.560537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:11:50.560537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:11:50.560537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:11:50.560537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:11:50.560537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:11:50.560537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:11:50.560537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:11:50.560537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:11:50.560537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:11:50.560537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:11:50.560537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:11:50.560537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:11:50.560537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:11:50.561553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:11:50.561553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:11:50.561553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:11:50.561553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:11:50.561553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:11:50.561553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:11:50.561553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:11:50.561553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:11:50.561553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:11:50.561553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:11:50.561553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:11:50.561553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:11:50.561553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 555/1101] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:11:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:11:50.567892 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833d1bd-9c1a-41b9-8d1a-d21f99820d7f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:50.571209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:50.682147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833d1bd-9c1a-41b9-8d1a-d21f99820d7f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:50.721912 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141526c-9b52-475b-9fdf-46c1711c00ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:50.722546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141526c-9b52-475b-9fdf-46c1711c00ae None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:50.722826 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141526c-9b52-475b-9fdf-46c1711c00ae None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:50.723092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141526c-9b52-475b-9fdf-46c1711c00ae None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:50.723815 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141526c-9b52-475b-9fdf-46c1711c00ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:50.724114 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141526c-9b52-475b-9fdf-46c1711c00ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:50.725508 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141526c-9b52-475b-9fdf-46c1711c00ae None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:50.734953 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 547/1102] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:11:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:50.744219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37dc24-8700-4c5d-adb6-a4d42850016b None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:50.862819 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37dc24-8700-4c5d-adb6-a4d42850016b None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:50.906857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149b916-2021-48ec-947c-61b05d2d071f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:50.907514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149b916-2021-48ec-947c-61b05d2d071f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:50.907797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149b916-2021-48ec-947c-61b05d2d071f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:50.908069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149b916-2021-48ec-947c-61b05d2d071f None admin] PATH_INFO: `/v3/users/f0f4cc15e3cf4ae98e5c6a6c7c86c8db` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:50.909391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149b916-2021-48ec-947c-61b05d2d071f None admin] RBAC: Authorizing `identity:get_user(user_id=f0f4cc15e3cf4ae98e5c6a6c7c86c8db)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:50.910878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149b916-2021-48ec-947c-61b05d2d071f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:50.912511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 556/1103] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:11:50 2022] GET /identity/v3/users/f0f4cc15e3cf4ae98e5c6a6c7c86c8db => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:50.921085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141526c-9b52-475b-9fdf-46c1711c00ae None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:51.000273 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:51.047574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141526c-9b52-475b-9fdf-46c1711c00ae None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:51.087819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d28535-2b63-4016-8aab-98c50f9e17e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:51.088443 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d28535-2b63-4016-8aab-98c50f9e17e4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:51.088718 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d28535-2b63-4016-8aab-98c50f9e17e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:51.088974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d28535-2b63-4016-8aab-98c50f9e17e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:51.159131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d28535-2b63-4016-8aab-98c50f9e17e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:51.160518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d28535-2b63-4016-8aab-98c50f9e17e4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:51.228248 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:51.275083 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 548/1104] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:11:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:11:51.283280 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149b916-2021-48ec-947c-61b05d2d071f None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:51.299148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:51.399507 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149b916-2021-48ec-947c-61b05d2d071f None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:51.435220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae275d5-40da-45c1-bf26-b638af9c3c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:51.435897 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae275d5-40da-45c1-bf26-b638af9c3c55 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:51.436176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae275d5-40da-45c1-bf26-b638af9c3c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:51.436441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae275d5-40da-45c1-bf26-b638af9c3c55 None admin] PATH_INFO: `/v3/projects/8db3c00959fe41f28504053f31faf49c` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:51.437632 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae275d5-40da-45c1-bf26-b638af9c3c55 None admin] RBAC: Authorizing `identity:get_project(project_id=8db3c00959fe41f28504053f31faf49c)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:51.439109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae275d5-40da-45c1-bf26-b638af9c3c55 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:51.440787 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 557/1105] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:11:51 2022] GET /identity/v3/projects/8db3c00959fe41f28504053f31faf49c => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:51.448135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d28535-2b63-4016-8aab-98c50f9e17e4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:51.566575 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d28535-2b63-4016-8aab-98c50f9e17e4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:51.603508 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bc20a-b6bf-484e-975e-1ff7b09b2ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:51.604120 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bc20a-b6bf-484e-975e-1ff7b09b2ebb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:51.604398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bc20a-b6bf-484e-975e-1ff7b09b2ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:51.604668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bc20a-b6bf-484e-975e-1ff7b09b2ebb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:51.605461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bc20a-b6bf-484e-975e-1ff7b09b2ebb None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=8db3c00959fe41f28504053f31faf49c, user.id=f0f4cc15e3cf4ae98e5c6a6c7c86c8db) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:51.605772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bc20a-b6bf-484e-975e-1ff7b09b2ebb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:51.607707 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bc20a-b6bf-484e-975e-1ff7b09b2ebb None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:51.614058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 549/1106] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:11:51 2022] GET /identity/v3/role_assignments?user.id=f0f4cc15e3cf4ae98e5c6a6c7c86c8db&scope.project.id=8db3c00959fe41f28504053f31faf49c&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:53.612943 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c335004-f2f1-4f01-97af-45044dc44e0b None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:53.613884 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c335004-f2f1-4f01-97af-45044dc44e0b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:53.613884 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c335004-f2f1-4f01-97af-45044dc44e0b None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:53.616039 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 558/1107] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:53.621375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a10c2-5d4f-4d0b-a74c-497702d39128 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:53.621818 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a10c2-5d4f-4d0b-a74c-497702d39128 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:53.621928 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a10c2-5d4f-4d0b-a74c-497702d39128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:53.640525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e24a10c2-5d4f-4d0b-a74c-497702d39128 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:53.641720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e24a10c2-5d4f-4d0b-a74c-497702d39128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:53.665693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:53.719747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 550/1108] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:53.749598 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c335004-f2f1-4f01-97af-45044dc44e0b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:53.840913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:53.871612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c335004-f2f1-4f01-97af-45044dc44e0b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:53.913332 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde5945-c801-4c8e-b1d3-06d7f5b3c0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:53.914002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde5945-c801-4c8e-b1d3-06d7f5b3c0fd None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:53.914290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde5945-c801-4c8e-b1d3-06d7f5b3c0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:53.914631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde5945-c801-4c8e-b1d3-06d7f5b3c0fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:53.915860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde5945-c801-4c8e-b1d3-06d7f5b3c0fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:53.921230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dde5945-c801-4c8e-b1d3-06d7f5b3c0fd None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:53.922879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 559/1109] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:11:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:53.929015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a10c2-5d4f-4d0b-a74c-497702d39128 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:54.034450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a10c2-5d4f-4d0b-a74c-497702d39128 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:54.069184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8621da0-2a44-4660-95e3-26e965daffa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:54.069816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8621da0-2a44-4660-95e3-26e965daffa3 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:54.070088 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8621da0-2a44-4660-95e3-26e965daffa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:54.070357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8621da0-2a44-4660-95e3-26e965daffa3 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:54.071024 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8621da0-2a44-4660-95e3-26e965daffa3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:54.072465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8621da0-2a44-4660-95e3-26e965daffa3 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:54.095315 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 551/1110] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:11:53 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:11:56.140001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b25092-8fc8-4a6f-b806-8998144420ec None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:56.140830 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b25092-8fc8-4a6f-b806-8998144420ec None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:56.140830 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b25092-8fc8-4a6f-b806-8998144420ec None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:56.142996 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 560/1111] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:56.148369 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265a0aa-6609-46d5-bd06-760f6131975f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:56.148809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265a0aa-6609-46d5-bd06-760f6131975f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:56.148912 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265a0aa-6609-46d5-bd06-760f6131975f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:56.169075 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6265a0aa-6609-46d5-bd06-760f6131975f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:56.170367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6265a0aa-6609-46d5-bd06-760f6131975f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:56.181765 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:56.248869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 552/1112] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:56.272246 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b25092-8fc8-4a6f-b806-8998144420ec None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:56.382108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b25092-8fc8-4a6f-b806-8998144420ec None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:56.418700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ea8e7-44dd-4ceb-a42d-0362fa8cf18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:56.419264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ea8e7-44dd-4ceb-a42d-0362fa8cf18a None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:56.419540 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ea8e7-44dd-4ceb-a42d-0362fa8cf18a None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:56.419839 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ea8e7-44dd-4ceb-a42d-0362fa8cf18a None admin] PATH_INFO: `/v3/roles/7ae6dccec1244987a5bc7bf5c3991703` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:56.421300 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ea8e7-44dd-4ceb-a42d-0362fa8cf18a None admin] RBAC: Authorizing `identity:get_role(role_id=7ae6dccec1244987a5bc7bf5c3991703)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:56.422703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ea8e7-44dd-4ceb-a42d-0362fa8cf18a None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:56.424076 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 561/1113] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:11:56 2022] GET /identity/v3/roles/7ae6dccec1244987a5bc7bf5c3991703 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:56.430639 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265a0aa-6609-46d5-bd06-760f6131975f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:56.538438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265a0aa-6609-46d5-bd06-760f6131975f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:56.575494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac11ab1-67cc-4627-8d00-ffc6a004c07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:56.576150 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac11ab1-67cc-4627-8d00-ffc6a004c07e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:56.576424 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac11ab1-67cc-4627-8d00-ffc6a004c07e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:56.576741 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac11ab1-67cc-4627-8d00-ffc6a004c07e None admin] PATH_INFO: `/v3/users/a09b1fe9945147d1a14e0b7f8bd2f59b` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:56.591426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac11ab1-67cc-4627-8d00-ffc6a004c07e None admin] RBAC: Authorizing `identity:get_user(user_id=a09b1fe9945147d1a14e0b7f8bd2f59b)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:56.592993 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac11ab1-67cc-4627-8d00-ffc6a004c07e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:56.594819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 553/1114] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:11:56 2022] GET /identity/v3/users/a09b1fe9945147d1a14e0b7f8bd2f59b => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:56.601721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ea8e7-44dd-4ceb-a42d-0362fa8cf18a None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:56.630006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:56.717954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ea8e7-44dd-4ceb-a42d-0362fa8cf18a None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:56.757961 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23dfb9-a1ea-434f-874a-841b8fd4c04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:56.758586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23dfb9-a1ea-434f-874a-841b8fd4c04a None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:56.758871 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23dfb9-a1ea-434f-874a-841b8fd4c04a None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:56.759144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23dfb9-a1ea-434f-874a-841b8fd4c04a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:56.833052 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c23dfb9-a1ea-434f-874a-841b8fd4c04a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:56.834591 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c23dfb9-a1ea-434f-874a-841b8fd4c04a None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:56.852277 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:56.949808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 562/1115] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:11:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:11:56.956984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac11ab1-67cc-4627-8d00-ffc6a004c07e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:56.961068 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:57.061225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac11ab1-67cc-4627-8d00-ffc6a004c07e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:57.097640 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981351f-5d71-4207-86b7-39f18c902edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:57.098344 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981351f-5d71-4207-86b7-39f18c902edc None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:57.098622 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981351f-5d71-4207-86b7-39f18c902edc None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:57.098894 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981351f-5d71-4207-86b7-39f18c902edc None admin] PATH_INFO: `/v3/projects/8db3c00959fe41f28504053f31faf49c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:57.100227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981351f-5d71-4207-86b7-39f18c902edc None admin] RBAC: Authorizing `identity:get_project(project_id=8db3c00959fe41f28504053f31faf49c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:57.101727 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981351f-5d71-4207-86b7-39f18c902edc None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:57.103503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 554/1116] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:11:56 2022] GET /identity/v3/projects/8db3c00959fe41f28504053f31faf49c => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:57.111962 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23dfb9-a1ea-434f-874a-841b8fd4c04a None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:57.224367 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23dfb9-a1ea-434f-874a-841b8fd4c04a None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:57.241301 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:57.260801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199616e5-a26c-4f82-a90f-c25c7b462b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:57.261408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199616e5-a26c-4f82-a90f-c25c7b462b8e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:57.261682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199616e5-a26c-4f82-a90f-c25c7b462b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:57.261973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199616e5-a26c-4f82-a90f-c25c7b462b8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:57.262763 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199616e5-a26c-4f82-a90f-c25c7b462b8e None admin] RBAC: Adding query filter params (role.id=7ae6dccec1244987a5bc7bf5c3991703, scope.project.id=8db3c00959fe41f28504053f31faf49c, user.id=a09b1fe9945147d1a14e0b7f8bd2f59b) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:11:57.263064 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199616e5-a26c-4f82-a90f-c25c7b462b8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:57.264510 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199616e5-a26c-4f82-a90f-c25c7b462b8e None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:57.270618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 563/1117] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:11:57 2022] GET /identity/v3/role_assignments?user.id=a09b1fe9945147d1a14e0b7f8bd2f59b&scope.project.id=8db3c00959fe41f28504053f31faf49c&role.id=7ae6dccec1244987a5bc7bf5c3991703 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:59.227498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86196100-0684-436a-b4ab-e434be529906 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:59.228298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86196100-0684-436a-b4ab-e434be529906 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:59.228298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86196100-0684-436a-b4ab-e434be529906 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:59.230268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 555/1118] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:11:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:11:59.236058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9ebb9-63e7-4e41-a2b0-f5ecc1dab104 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:59.236502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9ebb9-63e7-4e41-a2b0-f5ecc1dab104 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:59.236603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9ebb9-63e7-4e41-a2b0-f5ecc1dab104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:59.255971 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-88d9ebb9-63e7-4e41-a2b0-f5ecc1dab104 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:11:59.257145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-88d9ebb9-63e7-4e41-a2b0-f5ecc1dab104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:11:59.326919 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 564/1119] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:11:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:11:59.349845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86196100-0684-436a-b4ab-e434be529906 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:59.431119 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:59.465204 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86196100-0684-436a-b4ab-e434be529906 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:59.501966 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331a5f1-49a6-4381-a734-fffb478d1219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:59.502549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331a5f1-49a6-4381-a734-fffb478d1219 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:59.502829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331a5f1-49a6-4381-a734-fffb478d1219 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:59.503107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331a5f1-49a6-4381-a734-fffb478d1219 None admin] PATH_INFO: `/v3/roles/7ae6dccec1244987a5bc7bf5c3991703` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:59.504646 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331a5f1-49a6-4381-a734-fffb478d1219 None admin] RBAC: Authorizing `identity:get_role(role_id=7ae6dccec1244987a5bc7bf5c3991703)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:59.506070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331a5f1-49a6-4381-a734-fffb478d1219 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:59.507464 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 556/1120] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:11:59 2022] GET /identity/v3/roles/7ae6dccec1244987a5bc7bf5c3991703 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:59.516837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9ebb9-63e7-4e41-a2b0-f5ecc1dab104 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:59.618836 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9ebb9-63e7-4e41-a2b0-f5ecc1dab104 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:59.632476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:11:59.658665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57ed5b-836a-49ae-9496-bf4744826e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:59.659298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57ed5b-836a-49ae-9496-bf4744826e65 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:59.659575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57ed5b-836a-49ae-9496-bf4744826e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:59.659840 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57ed5b-836a-49ae-9496-bf4744826e65 None admin] PATH_INFO: `/v3/users/a09b1fe9945147d1a14e0b7f8bd2f59b` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:59.661000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57ed5b-836a-49ae-9496-bf4744826e65 None admin] RBAC: Authorizing `identity:get_user(user_id=a09b1fe9945147d1a14e0b7f8bd2f59b)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:59.662463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57ed5b-836a-49ae-9496-bf4744826e65 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:59.664186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 565/1121] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:11:59 2022] GET /identity/v3/users/a09b1fe9945147d1a14e0b7f8bd2f59b => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:59.670214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331a5f1-49a6-4381-a734-fffb478d1219 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:59.791692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331a5f1-49a6-4381-a734-fffb478d1219 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:59.833688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558cc72-6b4c-4e98-8596-79e147c5e022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:59.834345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558cc72-6b4c-4e98-8596-79e147c5e022 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:59.834617 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558cc72-6b4c-4e98-8596-79e147c5e022 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:59.834914 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558cc72-6b4c-4e98-8596-79e147c5e022 None admin] PATH_INFO: `/v3/projects/8db3c00959fe41f28504053f31faf49c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:59.836181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558cc72-6b4c-4e98-8596-79e147c5e022 None admin] RBAC: Authorizing `identity:get_project(project_id=8db3c00959fe41f28504053f31faf49c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:59.837625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558cc72-6b4c-4e98-8596-79e147c5e022 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:11:59.839297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 557/1122] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:11:59 2022] GET /identity/v3/projects/8db3c00959fe41f28504053f31faf49c => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:11:59.845566 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57ed5b-836a-49ae-9496-bf4744826e65 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:11:59.950748 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57ed5b-836a-49ae-9496-bf4744826e65 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:11:59.985149 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d846889-c1af-4d64-8bf4-354816fa9ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:11:59.985807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d846889-c1af-4d64-8bf4-354816fa9ba9 None admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:11:59.986104 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d846889-c1af-4d64-8bf4-354816fa9ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:11:59.986375 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d846889-c1af-4d64-8bf4-354816fa9ba9 None admin] PATH_INFO: `/v3/projects/8db3c00959fe41f28504053f31faf49c/users/a09b1fe9945147d1a14e0b7f8bd2f59b/roles/7ae6dccec1244987a5bc7bf5c3991703` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:11:59.988548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d846889-c1af-4d64-8bf4-354816fa9ba9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8db3c00959fe41f28504053f31faf49c, user_id=a09b1fe9945147d1a14e0b7f8bd2f59b, role_id=7ae6dccec1244987a5bc7bf5c3991703)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:11:59.990008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d846889-c1af-4d64-8bf4-354816fa9ba9 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:00.005868 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 566/1123] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:11:59 2022] PUT /identity/v3/projects/8db3c00959fe41f28504053f31faf49c/users/a09b1fe9945147d1a14e0b7f8bd2f59b/roles/7ae6dccec1244987a5bc7bf5c3991703 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:01.975347 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698c34c-31a8-43de-b1d2-e40ed03513ab None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:01.975926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698c34c-31a8-43de-b1d2-e40ed03513ab None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:01.975926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698c34c-31a8-43de-b1d2-e40ed03513ab None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:01.978211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 558/1124] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:12:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:12:01.987010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a94da8-d62a-4eb3-be3a-656ec09f71aa None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:01.987749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a94da8-d62a-4eb3-be3a-656ec09f71aa None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:01.987918 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a94da8-d62a-4eb3-be3a-656ec09f71aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:02.016202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-68a94da8-d62a-4eb3-be3a-656ec09f71aa None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:02.017420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-68a94da8-d62a-4eb3-be3a-656ec09f71aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:02.051620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:02.090095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 567/1125] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:12:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:12:02.115983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698c34c-31a8-43de-b1d2-e40ed03513ab None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:02.133157 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:02.235940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698c34c-31a8-43de-b1d2-e40ed03513ab None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:02.274101 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d130f-3e23-4c11-9f5e-f538d22b80ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:02.274750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d130f-3e23-4c11-9f5e-f538d22b80ae None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:02.275029 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d130f-3e23-4c11-9f5e-f538d22b80ae None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:02.275303 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d130f-3e23-4c11-9f5e-f538d22b80ae None admin] PATH_INFO: `/v3/roles/7ae6dccec1244987a5bc7bf5c3991703` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:02.276795 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d130f-3e23-4c11-9f5e-f538d22b80ae None admin] RBAC: Authorizing `identity:get_role(role_id=7ae6dccec1244987a5bc7bf5c3991703)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:02.278214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d130f-3e23-4c11-9f5e-f538d22b80ae None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:02.279616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 559/1126] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:12:02 2022] GET /identity/v3/roles/7ae6dccec1244987a5bc7bf5c3991703 => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:02.286593 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a94da8-d62a-4eb3-be3a-656ec09f71aa None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:02.413828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a94da8-d62a-4eb3-be3a-656ec09f71aa None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:02.452387 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fea0c8-c059-441f-b539-2b4c533a2ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:02.453029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fea0c8-c059-441f-b539-2b4c533a2ddf None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:02.453306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fea0c8-c059-441f-b539-2b4c533a2ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:02.453580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fea0c8-c059-441f-b539-2b4c533a2ddf None admin] PATH_INFO: `/v3/users/a09b1fe9945147d1a14e0b7f8bd2f59b` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:02.454851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fea0c8-c059-441f-b539-2b4c533a2ddf None admin] RBAC: Authorizing `identity:get_user(user_id=a09b1fe9945147d1a14e0b7f8bd2f59b)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:02.456310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fea0c8-c059-441f-b539-2b4c533a2ddf None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:02.457986 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 568/1127] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:12:02 2022] GET /identity/v3/users/a09b1fe9945147d1a14e0b7f8bd2f59b => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:02.466915 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d130f-3e23-4c11-9f5e-f538d22b80ae None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:02.551341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:02.579505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d130f-3e23-4c11-9f5e-f538d22b80ae None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:02.618885 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7db7d2-4709-46b4-a934-0c205ea747ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:02.619508 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7db7d2-4709-46b4-a934-0c205ea747ff None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:02.619786 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7db7d2-4709-46b4-a934-0c205ea747ff None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:02.620054 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7db7d2-4709-46b4-a934-0c205ea747ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:02.693852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7db7d2-4709-46b4-a934-0c205ea747ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:02.695323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7db7d2-4709-46b4-a934-0c205ea747ff None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:02.784043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:02.814810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 560/1128] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:12:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 15:12:02.821031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fea0c8-c059-441f-b539-2b4c533a2ddf None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:02.862979 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:02.928183 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fea0c8-c059-441f-b539-2b4c533a2ddf None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:02.963400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8c373-45dd-414a-af8f-ac6315790be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:02.963957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8c373-45dd-414a-af8f-ac6315790be6 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:02.964232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8c373-45dd-414a-af8f-ac6315790be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:02.964498 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8c373-45dd-414a-af8f-ac6315790be6 None admin] PATH_INFO: `/v3/projects/8db3c00959fe41f28504053f31faf49c` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:02.965755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8c373-45dd-414a-af8f-ac6315790be6 None admin] RBAC: Authorizing `identity:get_project(project_id=8db3c00959fe41f28504053f31faf49c)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:02.967261 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8c373-45dd-414a-af8f-ac6315790be6 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:02.968914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 569/1129] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:12:02 2022] GET /identity/v3/projects/8db3c00959fe41f28504053f31faf49c => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:02.976173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7db7d2-4709-46b4-a934-0c205ea747ff None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:03.092421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7db7d2-4709-46b4-a934-0c205ea747ff None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:03.128363 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814145cc-b783-44f9-81a7-8b4479ebd22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:03.129009 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814145cc-b783-44f9-81a7-8b4479ebd22b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:03.129325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814145cc-b783-44f9-81a7-8b4479ebd22b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:03.129638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814145cc-b783-44f9-81a7-8b4479ebd22b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:03.130528 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814145cc-b783-44f9-81a7-8b4479ebd22b None admin] RBAC: Adding query filter params (role.id=7ae6dccec1244987a5bc7bf5c3991703, scope.project.id=8db3c00959fe41f28504053f31faf49c, user.id=a09b1fe9945147d1a14e0b7f8bd2f59b) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:03.130862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814145cc-b783-44f9-81a7-8b4479ebd22b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:03.132350 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814145cc-b783-44f9-81a7-8b4479ebd22b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:03.139270 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 561/1130] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:12:02 2022] GET /identity/v3/role_assignments?user.id=a09b1fe9945147d1a14e0b7f8bd2f59b&scope.project.id=8db3c00959fe41f28504053f31faf49c&role.id=7ae6dccec1244987a5bc7bf5c3991703 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:05.130879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61087eba-da93-42d7-a72f-9e4a2b11bf50 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:05.131394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61087eba-da93-42d7-a72f-9e4a2b11bf50 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:05.131394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61087eba-da93-42d7-a72f-9e4a2b11bf50 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:05.133674 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 570/1131] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:12:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:12:05.141968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bdfe0-4497-4fd3-85a8-0a75cc276af1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:05.142665 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bdfe0-4497-4fd3-85a8-0a75cc276af1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:05.142963 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833bdfe0-4497-4fd3-85a8-0a75cc276af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:05.172256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-833bdfe0-4497-4fd3-85a8-0a75cc276af1 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:05.173498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-833bdfe0-4497-4fd3-85a8-0a75cc276af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:05.215827 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:05.251479 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 562/1132] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:12:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:12:05.274937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61087eba-da93-42d7-a72f-9e4a2b11bf50 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:05.385141 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61087eba-da93-42d7-a72f-9e4a2b11bf50 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:05.387430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:05.429659 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae88bc-4752-4c50-8657-a3512f1e0d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:05.430345 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae88bc-4752-4c50-8657-a3512f1e0d37 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:05.430612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae88bc-4752-4c50-8657-a3512f1e0d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:05.430883 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae88bc-4752-4c50-8657-a3512f1e0d37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:05.432096 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae88bc-4752-4c50-8657-a3512f1e0d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:05.433535 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae88bc-4752-4c50-8657-a3512f1e0d37 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:05.435173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 571/1133] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:12:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:05.445380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bdfe0-4497-4fd3-85a8-0a75cc276af1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:05.563477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833bdfe0-4497-4fd3-85a8-0a75cc276af1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:05.599584 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ba80f-cb49-4188-95ce-e1c530c930be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:05.600224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ba80f-cb49-4188-95ce-e1c530c930be None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:05.600560 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ba80f-cb49-4188-95ce-e1c530c930be None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:05.600851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ba80f-cb49-4188-95ce-e1c530c930be None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:05.601593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ba80f-cb49-4188-95ce-e1c530c930be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:05.603120 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ba80f-cb49-4188-95ce-e1c530c930be None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:05.637385 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 563/1134] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 15:12:05 2022] POST /identity/v3/projects => generated 311 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:07.670948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66a83a-b97e-432b-b1bd-e0108b7bd15c None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:07.671505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66a83a-b97e-432b-b1bd-e0108b7bd15c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:07.671505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66a83a-b97e-432b-b1bd-e0108b7bd15c None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:07.672838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 572/1135] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:12:07.678438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3161e-2db4-45ed-8b1d-bc78af3cc1d4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:07.678828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3161e-2db4-45ed-8b1d-bc78af3cc1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:07.679092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3161e-2db4-45ed-8b1d-bc78af3cc1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:07.698484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-59e3161e-2db4-45ed-8b1d-bc78af3cc1d4 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:07.699741 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-59e3161e-2db4-45ed-8b1d-bc78af3cc1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:07.717696 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:07.773488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 564/1136] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:12:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:12:07.799024 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66a83a-b97e-432b-b1bd-e0108b7bd15c None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:07.902054 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66a83a-b97e-432b-b1bd-e0108b7bd15c None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:07.938340 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9437eb12-2d4f-469d-9d7b-44f423b8b83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:07.938897 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437eb12-2d4f-469d-9d7b-44f423b8b83b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:07.939169 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437eb12-2d4f-469d-9d7b-44f423b8b83b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:07.939443 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437eb12-2d4f-469d-9d7b-44f423b8b83b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:07.940639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9437eb12-2d4f-469d-9d7b-44f423b8b83b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:07.942113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9437eb12-2d4f-469d-9d7b-44f423b8b83b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:07.943717 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 573/1137] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:12:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:07.949803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3161e-2db4-45ed-8b1d-bc78af3cc1d4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:08.053156 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3161e-2db4-45ed-8b1d-bc78af3cc1d4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:08.088107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0492b-8f04-4e79-8ca9-386762bb6d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:08.088738 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0492b-8f04-4e79-8ca9-386762bb6d57 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:08.089053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0492b-8f04-4e79-8ca9-386762bb6d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:08.089366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0492b-8f04-4e79-8ca9-386762bb6d57 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:08.090112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0492b-8f04-4e79-8ca9-386762bb6d57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:08.091600 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0492b-8f04-4e79-8ca9-386762bb6d57 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:08.114733 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 565/1138] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:12:07 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:10.118151 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f706b-d00c-45ac-aafe-e2f75fdd8635 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:10.118697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f706b-d00c-45ac-aafe-e2f75fdd8635 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:10.118697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f706b-d00c-45ac-aafe-e2f75fdd8635 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:10.120068 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 574/1139] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:12:10.125434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b797dc5-283c-4673-a213-cf97de84c69e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:10.125884 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b797dc5-283c-4673-a213-cf97de84c69e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:10.125997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b797dc5-283c-4673-a213-cf97de84c69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:10.144578 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5b797dc5-283c-4673-a213-cf97de84c69e None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:10.145680 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5b797dc5-283c-4673-a213-cf97de84c69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:10.147666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:10.222386 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 566/1140] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:12:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:12:10.246169 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f706b-d00c-45ac-aafe-e2f75fdd8635 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:10.292628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:10.360690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f706b-d00c-45ac-aafe-e2f75fdd8635 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:10.396871 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ba783-7409-49e0-a6e9-32f28c50f45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:10.397428 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ba783-7409-49e0-a6e9-32f28c50f45b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:10.397701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ba783-7409-49e0-a6e9-32f28c50f45b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:10.397991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ba783-7409-49e0-a6e9-32f28c50f45b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:10.408488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ba783-7409-49e0-a6e9-32f28c50f45b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:12:10.413342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-996ba783-7409-49e0-a6e9-32f28c50f45b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:12:10.413342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:12:10.413342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:12:10.413342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:12:10.413342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:12:10.413342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:12:10.413342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:12:10.413342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:12:10.413342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:12:10.413342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:12:10.413342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:12:10.413342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:12:10.413342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:12:10.413342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:12:10.413342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:12:10.413342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:12:10.414385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:12:10.414385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:12:10.414385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:12:10.414385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:12:10.414385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:12:10.414385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:12:10.414385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:12:10.414385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:12:10.414385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:12:10.414385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:12:10.414385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:12:10.414385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:12:10.414385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:12:10.414385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:12:10.414385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:12:10.414385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:12:10.415279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:12:10.415279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:12:10.415279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:12:10.415279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:12:10.415279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:12:10.415279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:12:10.415279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:12:10.415279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:12:10.415279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:12:10.415279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:12:10.415279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:12:10.415279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:12:10.415279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 575/1141] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:12:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:12:10.421060 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b797dc5-283c-4673-a213-cf97de84c69e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:10.538931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b797dc5-283c-4673-a213-cf97de84c69e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:10.579680 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b9702-ded7-4629-885d-a6aaadc4784f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:10.580484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b9702-ded7-4629-885d-a6aaadc4784f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:10.581003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b9702-ded7-4629-885d-a6aaadc4784f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:10.581520 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b9702-ded7-4629-885d-a6aaadc4784f None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:10.582844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b9702-ded7-4629-885d-a6aaadc4784f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:10.583220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b9702-ded7-4629-885d-a6aaadc4784f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:10.585226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b9702-ded7-4629-885d-a6aaadc4784f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:10.590939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:10.594816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 567/1142] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:12:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:10.600885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ba783-7409-49e0-a6e9-32f28c50f45b None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:10.700258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:10.703779 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ba783-7409-49e0-a6e9-32f28c50f45b None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:10.739005 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cdfaa-b033-42e9-8af5-5f48002d31a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:10.739630 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cdfaa-b033-42e9-8af5-5f48002d31a1 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:10.739904 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cdfaa-b033-42e9-8af5-5f48002d31a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:10.740136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cdfaa-b033-42e9-8af5-5f48002d31a1 None admin] PATH_INFO: `/v3/users/99142367f031445dae2ab4b3a3e83e4f` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:10.753814 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892cdfaa-b033-42e9-8af5-5f48002d31a1 None admin] RBAC: Authorizing `identity:get_user(user_id=99142367f031445dae2ab4b3a3e83e4f)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:10.755248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892cdfaa-b033-42e9-8af5-5f48002d31a1 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:10.756862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 576/1143] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:12:10 2022] GET /identity/v3/users/99142367f031445dae2ab4b3a3e83e4f => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:10.763800 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b9702-ded7-4629-885d-a6aaadc4784f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:10.867822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b9702-ded7-4629-885d-a6aaadc4784f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:10.907406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cfa57-dd11-4f28-9b59-6dedd99ef136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:10.908056 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cfa57-dd11-4f28-9b59-6dedd99ef136 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:10.908371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cfa57-dd11-4f28-9b59-6dedd99ef136 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:10.908671 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cfa57-dd11-4f28-9b59-6dedd99ef136 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:10.975914 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:10.978454 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64cfa57-dd11-4f28-9b59-6dedd99ef136 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:10.980049 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64cfa57-dd11-4f28-9b59-6dedd99ef136 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:11.081282 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 568/1144] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:12:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:12:11.087325 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cdfaa-b033-42e9-8af5-5f48002d31a1 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:11.195648 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cdfaa-b033-42e9-8af5-5f48002d31a1 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:11.231258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0149b7cb-ba62-4d34-9135-be58458b4e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:11.231832 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149b7cb-ba62-4d34-9135-be58458b4e0c None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:11.232109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149b7cb-ba62-4d34-9135-be58458b4e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:11.232423 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0149b7cb-ba62-4d34-9135-be58458b4e0c None admin] PATH_INFO: `/v3/projects/33b158f827fa427fbfacf71e3bcf87c1` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:11.233626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149b7cb-ba62-4d34-9135-be58458b4e0c None admin] RBAC: Authorizing `identity:get_project(project_id=33b158f827fa427fbfacf71e3bcf87c1)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:11.235105 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0149b7cb-ba62-4d34-9135-be58458b4e0c None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:11.236787 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 577/1145] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:12:11 2022] GET /identity/v3/projects/33b158f827fa427fbfacf71e3bcf87c1 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:11.243085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cfa57-dd11-4f28-9b59-6dedd99ef136 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:11.343287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:11.346528 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cfa57-dd11-4f28-9b59-6dedd99ef136 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:11.382440 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84adb45a-d499-4a4b-8250-48f909702123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:11.383218 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adb45a-d499-4a4b-8250-48f909702123 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:11.383692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adb45a-d499-4a4b-8250-48f909702123 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:11.384191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adb45a-d499-4a4b-8250-48f909702123 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:11.384997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adb45a-d499-4a4b-8250-48f909702123 None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=33b158f827fa427fbfacf71e3bcf87c1, user.id=99142367f031445dae2ab4b3a3e83e4f) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:11.385309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adb45a-d499-4a4b-8250-48f909702123 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:11.386794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adb45a-d499-4a4b-8250-48f909702123 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:11.392869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 569/1146] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:12:11 2022] GET /identity/v3/role_assignments?user.id=99142367f031445dae2ab4b3a3e83e4f&scope.project.id=33b158f827fa427fbfacf71e3bcf87c1&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:13.329076 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbfc13-fd4c-493a-9a87-e8f0aa0ffc8b None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:13.329597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbfc13-fd4c-493a-9a87-e8f0aa0ffc8b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:13.329597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbfc13-fd4c-493a-9a87-e8f0aa0ffc8b None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:13.331014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 578/1147] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:12:13.336561 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077ad86-9dc8-4f47-b79a-16b3ac9f1c3e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:13.336959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077ad86-9dc8-4f47-b79a-16b3ac9f1c3e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:13.337113 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077ad86-9dc8-4f47-b79a-16b3ac9f1c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:13.356285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a077ad86-9dc8-4f47-b79a-16b3ac9f1c3e None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:13.357604 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a077ad86-9dc8-4f47-b79a-16b3ac9f1c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:13.425948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 570/1148] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:12:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:12:13.448868 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbfc13-fd4c-493a-9a87-e8f0aa0ffc8b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:13.477209 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:13.554575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbfc13-fd4c-493a-9a87-e8f0aa0ffc8b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:13.590022 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c2d6d-ec3f-48f8-b986-688479830c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:13.590590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c2d6d-ec3f-48f8-b986-688479830c9e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:13.590863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c2d6d-ec3f-48f8-b986-688479830c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:13.591136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c2d6d-ec3f-48f8-b986-688479830c9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:13.601805 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c2d6d-ec3f-48f8-b986-688479830c9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:12:13.607145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-402c2d6d-ec3f-48f8-b986-688479830c9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:12:13.607145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:12:13.607145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:12:13.607145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:12:13.607145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:12:13.607145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:12:13.607145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:12:13.607145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:12:13.607145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:12:13.607145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:12:13.607145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:12:13.607145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:12:13.607145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:12:13.607145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:12:13.607145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:12:13.607145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:12:13.608208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:12:13.608208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:12:13.608208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:12:13.608208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:12:13.608208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:12:13.608208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:12:13.608208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:12:13.608208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:12:13.608208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:12:13.608208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:12:13.608208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:12:13.608208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:12:13.608208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:12:13.608208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:12:13.608208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:12:13.608208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:12:13.609083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:12:13.609083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:12:13.609083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:12:13.609083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:12:13.609083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:12:13.609083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:12:13.609083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:12:13.609083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:12:13.609083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:12:13.609083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:12:13.609083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:12:13.609083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:12:13.609083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 579/1149] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:12:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:12:13.614851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077ad86-9dc8-4f47-b79a-16b3ac9f1c3e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:13.715048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:13.722333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077ad86-9dc8-4f47-b79a-16b3ac9f1c3e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:13.757807 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9c52d-32ab-4508-9bb0-87bc700cba7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:13.758443 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9c52d-32ab-4508-9bb0-87bc700cba7f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:13.758722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9c52d-32ab-4508-9bb0-87bc700cba7f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:13.759006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9c52d-32ab-4508-9bb0-87bc700cba7f None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:13.759722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f9c52d-32ab-4508-9bb0-87bc700cba7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:13.760021 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f9c52d-32ab-4508-9bb0-87bc700cba7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:13.761409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f9c52d-32ab-4508-9bb0-87bc700cba7f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:13.770345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 571/1150] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:12:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:13.776548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c2d6d-ec3f-48f8-b986-688479830c9e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:13.862782 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:13.879352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c2d6d-ec3f-48f8-b986-688479830c9e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:13.919563 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d3421-d4e4-4cf3-a7cc-d87b4175425a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:13.920582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d3421-d4e4-4cf3-a7cc-d87b4175425a None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:13.921049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d3421-d4e4-4cf3-a7cc-d87b4175425a None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:13.922050 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d3421-d4e4-4cf3-a7cc-d87b4175425a None admin] PATH_INFO: `/v3/users/99142367f031445dae2ab4b3a3e83e4f` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:13.924351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d3421-d4e4-4cf3-a7cc-d87b4175425a None admin] RBAC: Authorizing `identity:get_user(user_id=99142367f031445dae2ab4b3a3e83e4f)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:13.926479 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d3421-d4e4-4cf3-a7cc-d87b4175425a None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:13.928556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 580/1151] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:12:13 2022] GET /identity/v3/users/99142367f031445dae2ab4b3a3e83e4f => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:13.936390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9c52d-32ab-4508-9bb0-87bc700cba7f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:14.042984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9c52d-32ab-4508-9bb0-87bc700cba7f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:14.077067 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c58a3-add8-4af8-a772-22e29c20df2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:14.077692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c58a3-add8-4af8-a772-22e29c20df2a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:14.077998 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c58a3-add8-4af8-a772-22e29c20df2a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:14.078267 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c58a3-add8-4af8-a772-22e29c20df2a None admin] PATH_INFO: `/v3/projects/33b158f827fa427fbfacf71e3bcf87c1` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:14.079448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c58a3-add8-4af8-a772-22e29c20df2a None admin] RBAC: Authorizing `identity:get_project(project_id=33b158f827fa427fbfacf71e3bcf87c1)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:14.080884 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c58a3-add8-4af8-a772-22e29c20df2a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:14.082593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 572/1152] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:12:13 2022] GET /identity/v3/projects/33b158f827fa427fbfacf71e3bcf87c1 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:14.089046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d3421-d4e4-4cf3-a7cc-d87b4175425a None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:14.195126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d3421-d4e4-4cf3-a7cc-d87b4175425a None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:14.232816 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccfa8d-65c0-4129-9c68-b80fcbdbb2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:14.233462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccfa8d-65c0-4129-9c68-b80fcbdbb2a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:14.233738 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccfa8d-65c0-4129-9c68-b80fcbdbb2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:14.234039 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccfa8d-65c0-4129-9c68-b80fcbdbb2a7 None admin] PATH_INFO: `/v3/projects/33b158f827fa427fbfacf71e3bcf87c1/users/99142367f031445dae2ab4b3a3e83e4f/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:14.236207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccfa8d-65c0-4129-9c68-b80fcbdbb2a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=33b158f827fa427fbfacf71e3bcf87c1, user_id=99142367f031445dae2ab4b3a3e83e4f, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:14.237655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccfa8d-65c0-4129-9c68-b80fcbdbb2a7 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:14.254354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 581/1153] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:12:14 2022] PUT /identity/v3/projects/33b158f827fa427fbfacf71e3bcf87c1/users/99142367f031445dae2ab4b3a3e83e4f/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:16.219275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678ea3a-1a54-413c-b851-11a6e0e6e859 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:16.220043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678ea3a-1a54-413c-b851-11a6e0e6e859 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:16.220043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678ea3a-1a54-413c-b851-11a6e0e6e859 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:16.221939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 573/1154] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:12:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:12:16.229302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6ac44-e343-4325-8b27-451768dd0c3f None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:16.229746 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6ac44-e343-4325-8b27-451768dd0c3f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:16.229860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6ac44-e343-4325-8b27-451768dd0c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:16.248408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-bbe6ac44-e343-4325-8b27-451768dd0c3f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:16.249611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-bbe6ac44-e343-4325-8b27-451768dd0c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:16.259175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:16.319268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 582/1155] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:12:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:12:16.342893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678ea3a-1a54-413c-b851-11a6e0e6e859 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:16.369880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:16.448671 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678ea3a-1a54-413c-b851-11a6e0e6e859 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:16.483416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed5571-810f-41d7-b6cd-a86e56e1f95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:16.483979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed5571-810f-41d7-b6cd-a86e56e1f95e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:16.484280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed5571-810f-41d7-b6cd-a86e56e1f95e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:16.484550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed5571-810f-41d7-b6cd-a86e56e1f95e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:16.495947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed5571-810f-41d7-b6cd-a86e56e1f95e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:12:16.503760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-d4ed5571-810f-41d7-b6cd-a86e56e1f95e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:12:16.503760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:12:16.503760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:12:16.503760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:12:16.503760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:12:16.503760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:12:16.503760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:12:16.503760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:12:16.503760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:12:16.503760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:12:16.503760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:12:16.503760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:12:16.503760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:12:16.503760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:12:16.503760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:12:16.503760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:12:16.505135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:12:16.505135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:12:16.505135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:12:16.505135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:12:16.505135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:12:16.505135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:12:16.505135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:12:16.505135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:12:16.505135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:12:16.505135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:12:16.505135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:12:16.505135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:12:16.505135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:12:16.505135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:12:16.505135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:12:16.505135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:12:16.506112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:12:16.506112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:12:16.506112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:12:16.506112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:12:16.506112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:12:16.506112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:12:16.506112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:12:16.506112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:12:16.506112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:12:16.506112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:12:16.506112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:12:16.506112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:12:16.506112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 574/1156] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:12:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:12:16.511862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6ac44-e343-4325-8b27-451768dd0c3f None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:16.624691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6ac44-e343-4325-8b27-451768dd0c3f None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:16.670532 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38355620-eb79-40d3-a19e-b5006fbba219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:16.671137 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38355620-eb79-40d3-a19e-b5006fbba219 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:16.671413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38355620-eb79-40d3-a19e-b5006fbba219 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:16.671683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38355620-eb79-40d3-a19e-b5006fbba219 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:16.672417 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38355620-eb79-40d3-a19e-b5006fbba219 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:16.672745 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38355620-eb79-40d3-a19e-b5006fbba219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:16.674158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38355620-eb79-40d3-a19e-b5006fbba219 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:16.682923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 583/1157] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:12:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:16.688925 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed5571-810f-41d7-b6cd-a86e56e1f95e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:16.774898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:16.791640 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed5571-810f-41d7-b6cd-a86e56e1f95e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:16.832301 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf6207-78da-4b95-8003-23fc1f895f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:16.832938 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf6207-78da-4b95-8003-23fc1f895f4f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:16.833214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf6207-78da-4b95-8003-23fc1f895f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:16.833488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf6207-78da-4b95-8003-23fc1f895f4f None admin] PATH_INFO: `/v3/users/99142367f031445dae2ab4b3a3e83e4f` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:16.834721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf6207-78da-4b95-8003-23fc1f895f4f None admin] RBAC: Authorizing `identity:get_user(user_id=99142367f031445dae2ab4b3a3e83e4f)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:16.836169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf6207-78da-4b95-8003-23fc1f895f4f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:16.837797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 575/1158] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:12:16 2022] GET /identity/v3/users/99142367f031445dae2ab4b3a3e83e4f => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:16.846913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38355620-eb79-40d3-a19e-b5006fbba219 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:16.856872 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:16.966781 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38355620-eb79-40d3-a19e-b5006fbba219 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:17.001166 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f38089-fece-4da0-8205-61b7ef724710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:17.001809 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f38089-fece-4da0-8205-61b7ef724710 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:17.002094 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f38089-fece-4da0-8205-61b7ef724710 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:17.002369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f38089-fece-4da0-8205-61b7ef724710 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:17.071775 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f38089-fece-4da0-8205-61b7ef724710 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:17.073241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f38089-fece-4da0-8205-61b7ef724710 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:17.081759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:17.178749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 584/1159] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:12:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:12:17.185050 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf6207-78da-4b95-8003-23fc1f895f4f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:17.294176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf6207-78da-4b95-8003-23fc1f895f4f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:17.327681 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f1e7f-321c-4f44-8f70-e47bfe669f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:17.328243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f1e7f-321c-4f44-8f70-e47bfe669f3f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:17.328523 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f1e7f-321c-4f44-8f70-e47bfe669f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:17.328795 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f1e7f-321c-4f44-8f70-e47bfe669f3f None admin] PATH_INFO: `/v3/projects/33b158f827fa427fbfacf71e3bcf87c1` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:17.329979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f1e7f-321c-4f44-8f70-e47bfe669f3f None admin] RBAC: Authorizing `identity:get_project(project_id=33b158f827fa427fbfacf71e3bcf87c1)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:17.331429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f1e7f-321c-4f44-8f70-e47bfe669f3f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:17.333131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 576/1160] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:12:17 2022] GET /identity/v3/projects/33b158f827fa427fbfacf71e3bcf87c1 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:17.339460 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f38089-fece-4da0-8205-61b7ef724710 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:17.445312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f38089-fece-4da0-8205-61b7ef724710 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:17.452770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:17.480765 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b39c34-dc93-4ad5-8fa6-18235b758084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:17.481383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b39c34-dc93-4ad5-8fa6-18235b758084 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:17.481658 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b39c34-dc93-4ad5-8fa6-18235b758084 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:17.481974 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b39c34-dc93-4ad5-8fa6-18235b758084 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:17.482764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b39c34-dc93-4ad5-8fa6-18235b758084 None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=33b158f827fa427fbfacf71e3bcf87c1, user.id=99142367f031445dae2ab4b3a3e83e4f) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:17.483080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b39c34-dc93-4ad5-8fa6-18235b758084 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:17.484518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b39c34-dc93-4ad5-8fa6-18235b758084 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:17.490780 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 585/1161] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:12:17 2022] GET /identity/v3/role_assignments?user.id=99142367f031445dae2ab4b3a3e83e4f&scope.project.id=33b158f827fa427fbfacf71e3bcf87c1&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:19.531917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6ca54-0f1b-4dac-9537-0900d1d17db5 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:19.532558 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6ca54-0f1b-4dac-9537-0900d1d17db5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:19.532558 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6ca54-0f1b-4dac-9537-0900d1d17db5 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:19.534814 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 577/1162] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:12:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:12:19.540600 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f6ce4-1e88-4124-bc2a-d24fbd579f54 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:19.541257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f6ce4-1e88-4124-bc2a-d24fbd579f54 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:19.541257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f6ce4-1e88-4124-bc2a-d24fbd579f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:19.561029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-f97f6ce4-1e88-4124-bc2a-d24fbd579f54 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:19.562280 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f97f6ce4-1e88-4124-bc2a-d24fbd579f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:19.634478 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 586/1163] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:12:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:12:19.663952 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6ca54-0f1b-4dac-9537-0900d1d17db5 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:19.692002 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:19.777135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6ca54-0f1b-4dac-9537-0900d1d17db5 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:19.812301 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff389530-f15a-4b66-a260-8dd65476cf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:19.812838 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff389530-f15a-4b66-a260-8dd65476cf17 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:19.813103 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff389530-f15a-4b66-a260-8dd65476cf17 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:19.813356 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff389530-f15a-4b66-a260-8dd65476cf17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:19.884918 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff389530-f15a-4b66-a260-8dd65476cf17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:19.886443 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff389530-f15a-4b66-a260-8dd65476cf17 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:19.907293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:19.993972 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 578/1164] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:12:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:12:20.000480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f6ce4-1e88-4124-bc2a-d24fbd579f54 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:20.112437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:20.115677 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f6ce4-1e88-4124-bc2a-d24fbd579f54 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:20.150471 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627969c-95bf-44ca-9005-d2208fcb23d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:20.151099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627969c-95bf-44ca-9005-d2208fcb23d3 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:20.151376 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627969c-95bf-44ca-9005-d2208fcb23d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:20.151650 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627969c-95bf-44ca-9005-d2208fcb23d3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:20.158409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627969c-95bf-44ca-9005-d2208fcb23d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:20.159879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627969c-95bf-44ca-9005-d2208fcb23d3 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-6627969c-95bf-44ca-9005-d2208fcb23d3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:12:20.171372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:12:20.172541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:12:20.172541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:12:20.172541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:12:20.172541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:12:20.172541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:12:20.172541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:12:20.172541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:12:20.172541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:12:20.172541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:12:20.172541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:12:20.172541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:12:20.172541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:12:20.172541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:12:20.172541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:12:20.172541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:12:20.172541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:12:20.173588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:12:20.173588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:12:20.173588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:12:20.173588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:12:20.173588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:12:20.173588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:12:20.173588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:12:20.173588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:12:20.173588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:12:20.173588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:12:20.173588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 01 15:12:20.173588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:12:20.173588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 15:12:20.173588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:12:20.173588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 587/1165] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 15:12:19 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:12:20.178934 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff389530-f15a-4b66-a260-8dd65476cf17 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:20.288915 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff389530-f15a-4b66-a260-8dd65476cf17 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:20.311647 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:20.327075 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505f5b6-9513-46c6-ba8d-557c63540235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:20.327737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505f5b6-9513-46c6-ba8d-557c63540235 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:20.328024 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505f5b6-9513-46c6-ba8d-557c63540235 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:20.328295 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505f5b6-9513-46c6-ba8d-557c63540235 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:20.329022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505f5b6-9513-46c6-ba8d-557c63540235 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:20.329331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505f5b6-9513-46c6-ba8d-557c63540235 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:20.330849 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505f5b6-9513-46c6-ba8d-557c63540235 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:20.341180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 579/1166] 10.222.0.48 () {56 vars in 1115 bytes} [Mon Aug 1 15:12:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:22.341202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325f3dc-1544-41a8-be08-8c928ac28e11 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:22.341948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325f3dc-1544-41a8-be08-8c928ac28e11 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:22.341948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b325f3dc-1544-41a8-be08-8c928ac28e11 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:22.343157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 588/1167] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:12:22.348557 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87d565-f845-4cb2-9493-12d45d29b748 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:22.349008 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87d565-f845-4cb2-9493-12d45d29b748 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:22.349105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87d565-f845-4cb2-9493-12d45d29b748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:22.368906 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ab87d565-f845-4cb2-9493-12d45d29b748 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:22.370144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ab87d565-f845-4cb2-9493-12d45d29b748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:22.441986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 580/1168] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:12:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:12:22.469876 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325f3dc-1544-41a8-be08-8c928ac28e11 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:22.577133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b325f3dc-1544-41a8-be08-8c928ac28e11 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:22.611515 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c1cc4-e24a-4448-a85f-405ceccba280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:22.612069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c1cc4-e24a-4448-a85f-405ceccba280 None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:22.612347 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c1cc4-e24a-4448-a85f-405ceccba280 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:22.612613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c1cc4-e24a-4448-a85f-405ceccba280 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:22.613336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c1cc4-e24a-4448-a85f-405ceccba280 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:22.614717 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c1cc4-e24a-4448-a85f-405ceccba280 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:22.631953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 589/1169] 10.222.0.48 () {60 vars in 1138 bytes} [Mon Aug 1 15:12:22 2022] POST /identity/v3/domains => generated 267 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:12:24.664118 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0952d6b-01ee-47a1-8933-f1744dbb9c08 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:24.664748 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0952d6b-01ee-47a1-8933-f1744dbb9c08 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:24.664748 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0952d6b-01ee-47a1-8933-f1744dbb9c08 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:24.666162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 581/1170] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:12:24.673064 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be8515-ef37-4400-aab9-b9aa6b90ff57 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:24.673616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be8515-ef37-4400-aab9-b9aa6b90ff57 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:24.673723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be8515-ef37-4400-aab9-b9aa6b90ff57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:24.702891 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-c8be8515-ef37-4400-aab9-b9aa6b90ff57 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:24.704619 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c8be8515-ef37-4400-aab9-b9aa6b90ff57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:24.722442 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:24.788823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 590/1171] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:12:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:12:24.825590 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0952d6b-01ee-47a1-8933-f1744dbb9c08 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:24.972681 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0952d6b-01ee-47a1-8933-f1744dbb9c08 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:24.988621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:25.015669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577abf3-afc2-4c8d-a18d-968b3f7c4e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:25.016346 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577abf3-afc2-4c8d-a18d-968b3f7c4e3e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:25.016627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577abf3-afc2-4c8d-a18d-968b3f7c4e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:25.016909 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577abf3-afc2-4c8d-a18d-968b3f7c4e3e None admin] PATH_INFO: `/v3/domains/88ac0b2d855c4809ac3333c70e2ed1db` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:25.029006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577abf3-afc2-4c8d-a18d-968b3f7c4e3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=88ac0b2d855c4809ac3333c70e2ed1db)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:25.030547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577abf3-afc2-4c8d-a18d-968b3f7c4e3e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:25.032165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 582/1172] 10.222.0.48 () {56 vars in 1183 bytes} [Mon Aug 1 15:12:24 2022] GET /identity/v3/domains/88ac0b2d855c4809ac3333c70e2ed1db => generated 267 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:25.039329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be8515-ef37-4400-aab9-b9aa6b90ff57 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:25.162062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be8515-ef37-4400-aab9-b9aa6b90ff57 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:25.206140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56bbcc-eae7-42bb-89e4-72503e877958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:25.206779 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56bbcc-eae7-42bb-89e4-72503e877958 None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:25.207058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56bbcc-eae7-42bb-89e4-72503e877958 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:25.207468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56bbcc-eae7-42bb-89e4-72503e877958 None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:25.208181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56bbcc-eae7-42bb-89e4-72503e877958 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:25.209691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56bbcc-eae7-42bb-89e4-72503e877958 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:25.241483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 591/1173] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 15:12:25 2022] POST /identity/v3/projects => generated 361 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:12:27.305422 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ee498-b19b-4a05-a4ca-8bc9efead282 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:27.306183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ee498-b19b-4a05-a4ca-8bc9efead282 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:27.306183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ee498-b19b-4a05-a4ca-8bc9efead282 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:27.307728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 583/1174] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:12:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:12:27.314788 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb6725-dea3-46ff-8177-643e2b406d22 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:27.315590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb6725-dea3-46ff-8177-643e2b406d22 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:27.315590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb6725-dea3-46ff-8177-643e2b406d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:27.322714 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:27.339020 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-0cdb6725-dea3-46ff-8177-643e2b406d22 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:27.340533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-0cdb6725-dea3-46ff-8177-643e2b406d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:27.420096 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 592/1175] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:12:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:12:27.448819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ee498-b19b-4a05-a4ca-8bc9efead282 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:27.569810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ee498-b19b-4a05-a4ca-8bc9efead282 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:27.605160 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cceee5-e7fc-491a-8b59-c17ca49143b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:27.606192 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cceee5-e7fc-491a-8b59-c17ca49143b8 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:27.606467 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cceee5-e7fc-491a-8b59-c17ca49143b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:27.606774 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cceee5-e7fc-491a-8b59-c17ca49143b8 None admin] PATH_INFO: `/v3/domains/88ac0b2d855c4809ac3333c70e2ed1db` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:27.608023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cceee5-e7fc-491a-8b59-c17ca49143b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=88ac0b2d855c4809ac3333c70e2ed1db)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:27.609651 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cceee5-e7fc-491a-8b59-c17ca49143b8 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:27.611278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 584/1176] 10.222.0.48 () {56 vars in 1183 bytes} [Mon Aug 1 15:12:27 2022] GET /identity/v3/domains/88ac0b2d855c4809ac3333c70e2ed1db => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:27.617703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb6725-dea3-46ff-8177-643e2b406d22 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:27.720367 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb6725-dea3-46ff-8177-643e2b406d22 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:27.752077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:27.757871 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320424ef-4aa7-4ff7-a0bd-46b06706dc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:27.758521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320424ef-4aa7-4ff7-a0bd-46b06706dc1d None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:27.758796 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320424ef-4aa7-4ff7-a0bd-46b06706dc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:27.759072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320424ef-4aa7-4ff7-a0bd-46b06706dc1d None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:27.759784 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320424ef-4aa7-4ff7-a0bd-46b06706dc1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:27.761321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320424ef-4aa7-4ff7-a0bd-46b06706dc1d None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:27.786668 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 593/1177] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:12:27 2022] POST /identity/v3/users => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:12:29.837627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85597f7-2a64-4572-a863-aeb46a2d8fe7 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:29.838328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85597f7-2a64-4572-a863-aeb46a2d8fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:29.838328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85597f7-2a64-4572-a863-aeb46a2d8fe7 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:29.839624 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 585/1178] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:12:29.845255 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae6f5e-3b69-4efb-88fd-76468ed614d6 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:29.845717 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae6f5e-3b69-4efb-88fd-76468ed614d6 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:29.845843 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae6f5e-3b69-4efb-88fd-76468ed614d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:29.866797 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-24ae6f5e-3b69-4efb-88fd-76468ed614d6 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:29.868077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-24ae6f5e-3b69-4efb-88fd-76468ed614d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:29.947367 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 594/1179] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:12:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:12:29.972806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85597f7-2a64-4572-a863-aeb46a2d8fe7 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.032043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:30.101344 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85597f7-2a64-4572-a863-aeb46a2d8fe7 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:30.137835 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2bb30-2ba7-4124-8ebf-97775ae08476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:30.138525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2bb30-2ba7-4124-8ebf-97775ae08476 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.138803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2bb30-2ba7-4124-8ebf-97775ae08476 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.139107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2bb30-2ba7-4124-8ebf-97775ae08476 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.150787 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2bb30-2ba7-4124-8ebf-97775ae08476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:12:30.155780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-abb2bb30-2ba7-4124-8ebf-97775ae08476 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:12:30.155780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:12:30.155780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:12:30.155780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:12:30.155780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:12:30.155780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:12:30.155780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:12:30.155780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:12:30.155780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:12:30.155780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:12:30.155780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:12:30.155780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:12:30.155780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:12:30.155780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:12:30.155780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:12:30.155780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:12:30.156999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:12:30.156999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:12:30.156999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:12:30.156999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:12:30.156999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:12:30.156999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:12:30.156999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:12:30.156999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:12:30.156999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:12:30.156999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:12:30.156999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:12:30.156999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:12:30.156999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:12:30.156999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:12:30.156999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:12:30.156999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:12:30.158268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:12:30.158268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:12:30.158268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:12:30.158268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:12:30.158268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:12:30.158268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:12:30.158268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:12:30.158268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:12:30.158268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:12:30.158268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:12:30.158268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:12:30.158268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:12:30.158268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 586/1180] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:12:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:12:30.163537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae6f5e-3b69-4efb-88fd-76468ed614d6 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.272628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae6f5e-3b69-4efb-88fd-76468ed614d6 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:30.291088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:30.310726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95323d62-95b5-4d95-8ac6-f902db9ea726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:30.311440 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95323d62-95b5-4d95-8ac6-f902db9ea726 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.311714 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95323d62-95b5-4d95-8ac6-f902db9ea726 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.311968 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95323d62-95b5-4d95-8ac6-f902db9ea726 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.312701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95323d62-95b5-4d95-8ac6-f902db9ea726 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:30.313001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95323d62-95b5-4d95-8ac6-f902db9ea726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:30.315202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95323d62-95b5-4d95-8ac6-f902db9ea726 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:30.325315 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 595/1181] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:12:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:30.331362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2bb30-2ba7-4124-8ebf-97775ae08476 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.445071 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2bb30-2ba7-4124-8ebf-97775ae08476 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:30.446582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:30.482461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb355c5-4b52-4d68-99e1-0c64105cf394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:30.483126 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb355c5-4b52-4d68-99e1-0c64105cf394 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.483409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb355c5-4b52-4d68-99e1-0c64105cf394 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.483684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb355c5-4b52-4d68-99e1-0c64105cf394 None admin] PATH_INFO: `/v3/users/c2c10b42f9554cd9b25a864df17651e1` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.496943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb355c5-4b52-4d68-99e1-0c64105cf394 None admin] RBAC: Authorizing `identity:get_user(user_id=c2c10b42f9554cd9b25a864df17651e1)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:30.498477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb355c5-4b52-4d68-99e1-0c64105cf394 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:30.500165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 587/1182] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:12:30 2022] GET /identity/v3/users/c2c10b42f9554cd9b25a864df17651e1 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:30.507399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95323d62-95b5-4d95-8ac6-f902db9ea726 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.623724 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95323d62-95b5-4d95-8ac6-f902db9ea726 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:30.663321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656a2b3-6577-4c44-9da1-f8c7151f7828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:30.663951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656a2b3-6577-4c44-9da1-f8c7151f7828 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:30.664231 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656a2b3-6577-4c44-9da1-f8c7151f7828 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:30.664503 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656a2b3-6577-4c44-9da1-f8c7151f7828 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:30.719393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:30.741810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e656a2b3-6577-4c44-9da1-f8c7151f7828 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:30.743294 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e656a2b3-6577-4c44-9da1-f8c7151f7828 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:30.865772 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 596/1183] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:12:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:12:30.872069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb355c5-4b52-4d68-99e1-0c64105cf394 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:30.981544 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb355c5-4b52-4d68-99e1-0c64105cf394 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.020487 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7107fd-b5dd-4930-9964-2c055eb169da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.021167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7107fd-b5dd-4930-9964-2c055eb169da None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.021447 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7107fd-b5dd-4930-9964-2c055eb169da None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.021723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7107fd-b5dd-4930-9964-2c055eb169da None admin] PATH_INFO: `/v3/projects/0f49196ec12f4d539079c0e003d7e380` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.022957 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7107fd-b5dd-4930-9964-2c055eb169da None admin] RBAC: Authorizing `identity:get_project(project_id=0f49196ec12f4d539079c0e003d7e380)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.024414 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7107fd-b5dd-4930-9964-2c055eb169da None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.026158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 588/1184] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:12:30 2022] GET /identity/v3/projects/0f49196ec12f4d539079c0e003d7e380 => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:31.032741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656a2b3-6577-4c44-9da1-f8c7151f7828 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:31.116794 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:31.141459 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656a2b3-6577-4c44-9da1-f8c7151f7828 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:31.179854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec349e84-3cec-4c06-9ef2-2b366d4b8ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:31.180466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec349e84-3cec-4c06-9ef2-2b366d4b8ddc None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:31.180738 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec349e84-3cec-4c06-9ef2-2b366d4b8ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:31.181003 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec349e84-3cec-4c06-9ef2-2b366d4b8ddc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:31.181825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec349e84-3cec-4c06-9ef2-2b366d4b8ddc None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=0f49196ec12f4d539079c0e003d7e380, user.id=c2c10b42f9554cd9b25a864df17651e1) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:31.182127 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec349e84-3cec-4c06-9ef2-2b366d4b8ddc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:31.183555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec349e84-3cec-4c06-9ef2-2b366d4b8ddc None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:31.189178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 597/1185] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:12:31 2022] GET /identity/v3/role_assignments?user.id=c2c10b42f9554cd9b25a864df17651e1&scope.project.id=0f49196ec12f4d539079c0e003d7e380&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:33.177522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e93604-fd12-48df-9283-d034a67496b5 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.178400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e93604-fd12-48df-9283-d034a67496b5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.178400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e93604-fd12-48df-9283-d034a67496b5 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.180574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 589/1186] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:12:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:12:33.186939 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09ca13-1e3a-4403-bcfd-a62bb8531d3f None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.187379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09ca13-1e3a-4403-bcfd-a62bb8531d3f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.187488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09ca13-1e3a-4403-bcfd-a62bb8531d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.211890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-6f09ca13-1e3a-4403-bcfd-a62bb8531d3f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:33.213092 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6f09ca13-1e3a-4403-bcfd-a62bb8531d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:33.287709 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 598/1187] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:12:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:12:33.310959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e93604-fd12-48df-9283-d034a67496b5 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:33.344302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:33.431340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e93604-fd12-48df-9283-d034a67496b5 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:33.468990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b574fa-2747-403d-bf64-9220e6ea5e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:33.469634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b574fa-2747-403d-bf64-9220e6ea5e8e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.469934 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b574fa-2747-403d-bf64-9220e6ea5e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.470215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b574fa-2747-403d-bf64-9220e6ea5e8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.481184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b574fa-2747-403d-bf64-9220e6ea5e8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:12:33.486256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-b8b574fa-2747-403d-bf64-9220e6ea5e8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:12:33.486256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:12:33.486256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:12:33.486256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:12:33.486256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:12:33.486256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:12:33.486256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:12:33.486256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:12:33.486256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:12:33.486256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:12:33.486256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:12:33.486256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:12:33.486256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:12:33.486256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:12:33.486256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:12:33.486256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:12:33.487320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:12:33.487320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:12:33.487320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:12:33.487320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:12:33.487320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:12:33.487320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:12:33.487320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:12:33.487320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:12:33.487320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:12:33.487320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:12:33.487320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:12:33.487320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:12:33.487320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:12:33.487320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:12:33.487320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:12:33.487320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:12:33.488999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:12:33.488999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:12:33.488999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:12:33.488999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:12:33.488999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:12:33.488999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:12:33.488999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:12:33.488999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:12:33.488999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:12:33.488999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:12:33.488999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:12:33.488999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:12:33.488999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 590/1188] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:12:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:12:33.497229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09ca13-1e3a-4403-bcfd-a62bb8531d3f None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:33.588812 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:33.618496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09ca13-1e3a-4403-bcfd-a62bb8531d3f None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:33.657244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f8acd-cf69-46df-ad92-947af17ffd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:33.657960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f8acd-cf69-46df-ad92-947af17ffd3b None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.658213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f8acd-cf69-46df-ad92-947af17ffd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.658517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f8acd-cf69-46df-ad92-947af17ffd3b None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.659250 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f8acd-cf69-46df-ad92-947af17ffd3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:33.659557 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f8acd-cf69-46df-ad92-947af17ffd3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:33.660940 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f8acd-cf69-46df-ad92-947af17ffd3b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:33.670673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 599/1189] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:12:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:33.679248 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b574fa-2747-403d-bf64-9220e6ea5e8e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:33.774562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:33.788507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b574fa-2747-403d-bf64-9220e6ea5e8e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:33.825736 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48197f-2f24-4e0f-8968-0bd42430645a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:33.826396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48197f-2f24-4e0f-8968-0bd42430645a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.826673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48197f-2f24-4e0f-8968-0bd42430645a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.826943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48197f-2f24-4e0f-8968-0bd42430645a None admin] PATH_INFO: `/v3/users/c2c10b42f9554cd9b25a864df17651e1` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.828147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48197f-2f24-4e0f-8968-0bd42430645a None admin] RBAC: Authorizing `identity:get_user(user_id=c2c10b42f9554cd9b25a864df17651e1)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:33.829668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48197f-2f24-4e0f-8968-0bd42430645a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:33.831307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 591/1190] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:12:33 2022] GET /identity/v3/users/c2c10b42f9554cd9b25a864df17651e1 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:33.841001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f8acd-cf69-46df-ad92-947af17ffd3b None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:33.962455 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f8acd-cf69-46df-ad92-947af17ffd3b None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:34.004880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4016c7-6614-4b46-a537-f7956e31aeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:34.005521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4016c7-6614-4b46-a537-f7956e31aeb3 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.005825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4016c7-6614-4b46-a537-f7956e31aeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.006095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4016c7-6614-4b46-a537-f7956e31aeb3 None admin] PATH_INFO: `/v3/projects/0f49196ec12f4d539079c0e003d7e380` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.007298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4016c7-6614-4b46-a537-f7956e31aeb3 None admin] RBAC: Authorizing `identity:get_project(project_id=0f49196ec12f4d539079c0e003d7e380)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:34.008877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4016c7-6614-4b46-a537-f7956e31aeb3 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:34.010629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 600/1191] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:12:33 2022] GET /identity/v3/projects/0f49196ec12f4d539079c0e003d7e380 => generated 361 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:34.018790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48197f-2f24-4e0f-8968-0bd42430645a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:34.134688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48197f-2f24-4e0f-8968-0bd42430645a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:34.172788 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbaad8-3db5-47a6-b4c6-8445ab0b5f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:34.173492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbaad8-3db5-47a6-b4c6-8445ab0b5f5e None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.173772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbaad8-3db5-47a6-b4c6-8445ab0b5f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.174077 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbaad8-3db5-47a6-b4c6-8445ab0b5f5e None admin] PATH_INFO: `/v3/projects/0f49196ec12f4d539079c0e003d7e380/users/c2c10b42f9554cd9b25a864df17651e1/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.176393 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbaad8-3db5-47a6-b4c6-8445ab0b5f5e None admin] RBAC: Authorizing `identity:create_grant(project_id=0f49196ec12f4d539079c0e003d7e380, user_id=c2c10b42f9554cd9b25a864df17651e1, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:34.177929 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bbaad8-3db5-47a6-b4c6-8445ab0b5f5e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:34.193939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 592/1192] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:12:34 2022] PUT /identity/v3/projects/0f49196ec12f4d539079c0e003d7e380/users/c2c10b42f9554cd9b25a864df17651e1/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:12:36.173512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb694d-0f96-4c6c-b1dc-ae06d0efd470 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:36.174515 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb694d-0f96-4c6c-b1dc-ae06d0efd470 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:36.174515 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb694d-0f96-4c6c-b1dc-ae06d0efd470 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:36.176708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 601/1193] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:12:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:12:36.186256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024b6c5-2129-43b6-9a0d-47374b4a84a5 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:36.186995 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024b6c5-2129-43b6-9a0d-47374b4a84a5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:36.187191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024b6c5-2129-43b6-9a0d-47374b4a84a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:36.213079 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a024b6c5-2129-43b6-9a0d-47374b4a84a5 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:36.214319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a024b6c5-2129-43b6-9a0d-47374b4a84a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:36.225498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:36.287896 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 593/1194] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:12:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:12:36.311518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb694d-0f96-4c6c-b1dc-ae06d0efd470 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:36.317530 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:36.441104 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb694d-0f96-4c6c-b1dc-ae06d0efd470 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:36.481295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad742a-3755-4bea-84b4-cccdcb2492ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:36.481980 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad742a-3755-4bea-84b4-cccdcb2492ef None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:36.482256 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad742a-3755-4bea-84b4-cccdcb2492ef None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:36.482526 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad742a-3755-4bea-84b4-cccdcb2492ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:36.494045 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad742a-3755-4bea-84b4-cccdcb2492ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:12:36.499080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-baad742a-3755-4bea-84b4-cccdcb2492ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:12:36.499080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:12:36.499080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:12:36.499080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:12:36.499080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:12:36.499080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:12:36.499080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:12:36.499080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:12:36.499080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:12:36.499080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:12:36.499080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:12:36.499080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:12:36.499080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:12:36.499080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:12:36.499080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:12:36.499080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:12:36.500211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:12:36.500211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:12:36.500211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:12:36.500211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:12:36.500211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:12:36.500211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:12:36.500211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:12:36.500211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:12:36.500211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:12:36.500211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:12:36.500211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:12:36.500211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:12:36.500211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:12:36.500211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:12:36.500211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:12:36.500211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:12:36.501227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:12:36.501227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:12:36.501227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:12:36.501227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:12:36.501227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:12:36.501227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:12:36.501227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:12:36.501227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:12:36.501227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:12:36.501227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:12:36.501227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:12:36.501227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:12:36.501227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 602/1195] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:12:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:12:36.507325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024b6c5-2129-43b6-9a0d-47374b4a84a5 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:36.617100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024b6c5-2129-43b6-9a0d-47374b4a84a5 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:36.654081 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288089d2-52cf-4a05-9e79-9a48300b046a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:36.654716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288089d2-52cf-4a05-9e79-9a48300b046a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:36.655001 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288089d2-52cf-4a05-9e79-9a48300b046a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:36.655289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288089d2-52cf-4a05-9e79-9a48300b046a None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:36.655980 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288089d2-52cf-4a05-9e79-9a48300b046a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:36.656280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288089d2-52cf-4a05-9e79-9a48300b046a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:36.657869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288089d2-52cf-4a05-9e79-9a48300b046a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:36.667176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 594/1196] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:12:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:36.676929 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad742a-3755-4bea-84b4-cccdcb2492ef None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:36.748989 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:36.795575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad742a-3755-4bea-84b4-cccdcb2492ef None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:36.850893 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f1260-86cf-42e1-a900-72f2e6020d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:36.851836 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f1260-86cf-42e1-a900-72f2e6020d66 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:36.852247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f1260-86cf-42e1-a900-72f2e6020d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:36.852642 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f1260-86cf-42e1-a900-72f2e6020d66 None admin] PATH_INFO: `/v3/users/c2c10b42f9554cd9b25a864df17651e1` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:36.854615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f1260-86cf-42e1-a900-72f2e6020d66 None admin] RBAC: Authorizing `identity:get_user(user_id=c2c10b42f9554cd9b25a864df17651e1)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:36.857245 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f1260-86cf-42e1-a900-72f2e6020d66 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:36.859824 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 603/1197] 10.222.0.48 () {56 vars in 1177 bytes} [Mon Aug 1 15:12:36 2022] GET /identity/v3/users/c2c10b42f9554cd9b25a864df17651e1 => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:36.866422 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288089d2-52cf-4a05-9e79-9a48300b046a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:36.879396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:36.972717 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288089d2-52cf-4a05-9e79-9a48300b046a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:37.015803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f014fd-c412-4e81-a6d6-f73ab46f01c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:37.016536 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f014fd-c412-4e81-a6d6-f73ab46f01c2 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:37.016896 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f014fd-c412-4e81-a6d6-f73ab46f01c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:37.017346 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f014fd-c412-4e81-a6d6-f73ab46f01c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:37.092255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f014fd-c412-4e81-a6d6-f73ab46f01c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:37.093721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f014fd-c412-4e81-a6d6-f73ab46f01c2 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:37.103794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:37.210311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 595/1198] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:12:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:12:37.216425 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f1260-86cf-42e1-a900-72f2e6020d66 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:37.331637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f1260-86cf-42e1-a900-72f2e6020d66 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:37.363406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:37.372287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90f2e5-dcbb-4592-a998-7afc6375c868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:37.373281 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90f2e5-dcbb-4592-a998-7afc6375c868 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:37.373688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90f2e5-dcbb-4592-a998-7afc6375c868 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:37.374142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90f2e5-dcbb-4592-a998-7afc6375c868 None admin] PATH_INFO: `/v3/projects/0f49196ec12f4d539079c0e003d7e380` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:37.376207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90f2e5-dcbb-4592-a998-7afc6375c868 None admin] RBAC: Authorizing `identity:get_project(project_id=0f49196ec12f4d539079c0e003d7e380)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:37.377851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90f2e5-dcbb-4592-a998-7afc6375c868 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:37.379645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 604/1199] 10.222.0.48 () {56 vars in 1186 bytes} [Mon Aug 1 15:12:37 2022] GET /identity/v3/projects/0f49196ec12f4d539079c0e003d7e380 => generated 361 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:37.387048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f014fd-c412-4e81-a6d6-f73ab46f01c2 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:37.517400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f014fd-c412-4e81-a6d6-f73ab46f01c2 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:37.528345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:37.557092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be416cf1-a150-4e59-8e3d-938a21b9de26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:37.557727 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be416cf1-a150-4e59-8e3d-938a21b9de26 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:37.558031 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be416cf1-a150-4e59-8e3d-938a21b9de26 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:37.558300 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be416cf1-a150-4e59-8e3d-938a21b9de26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:37.559095 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be416cf1-a150-4e59-8e3d-938a21b9de26 None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=0f49196ec12f4d539079c0e003d7e380, user.id=c2c10b42f9554cd9b25a864df17651e1) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:12:37.559397 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be416cf1-a150-4e59-8e3d-938a21b9de26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:37.560875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be416cf1-a150-4e59-8e3d-938a21b9de26 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:37.567123 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 596/1200] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:12:37 2022] GET /identity/v3/role_assignments?user.id=c2c10b42f9554cd9b25a864df17651e1&scope.project.id=0f49196ec12f4d539079c0e003d7e380&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:47.564191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=14 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:33.058486 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8fe70-3e4e-4b40-ad56-7c656f7d7fe6 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:33.059091 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8fe70-3e4e-4b40-ad56-7c656f7d7fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:33.059091 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8fe70-3e4e-4b40-ad56-7c656f7d7fe6 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:33.060766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 605/1201] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:13:33.066394 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c911876-1135-49aa-987b-b2cd2e6abe9c None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:33.067299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c911876-1135-49aa-987b-b2cd2e6abe9c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:33.067787 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c911876-1135-49aa-987b-b2cd2e6abe9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:33.088270 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5c911876-1135-49aa-987b-b2cd2e6abe9c None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:33.089539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5c911876-1135-49aa-987b-b2cd2e6abe9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:33.181941 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 597/1202] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:13:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:13:33.209023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8fe70-3e4e-4b40-ad56-7c656f7d7fe6 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:33.345556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8fe70-3e4e-4b40-ad56-7c656f7d7fe6 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:33.391551 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528563d4-5e2e-46f9-b83f-8ccfa231dbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:33.392213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528563d4-5e2e-46f9-b83f-8ccfa231dbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:33.392551 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528563d4-5e2e-46f9-b83f-8ccfa231dbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:33.392759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528563d4-5e2e-46f9-b83f-8ccfa231dbc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:33.407308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528563d4-5e2e-46f9-b83f-8ccfa231dbc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:33.408877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528563d4-5e2e-46f9-b83f-8ccfa231dbc8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:33.428108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-528563d4-5e2e-46f9-b83f-8ccfa231dbc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:33.428108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:33.428108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:33.428108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:33.428108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:33.428108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:33.428108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:33.428108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:33.428108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:33.428108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:33.428108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:33.428108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:33.428108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:13:33.428108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:13:33.428108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:33.429259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:33.430313 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:13:33.430313 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:13:33.430313 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:33.430313 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:13:33.437268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 606/1203] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:33.437456 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c911876-1135-49aa-987b-b2cd2e6abe9c None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:33.557720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c911876-1135-49aa-987b-b2cd2e6abe9c None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:33.597682 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:33.606174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa9f398-f02a-460b-8973-4073eac1f919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:33.607033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9f398-f02a-460b-8973-4073eac1f919 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:33.607311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9f398-f02a-460b-8973-4073eac1f919 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:33.607579 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa9f398-f02a-460b-8973-4073eac1f919 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:33.608275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9f398-f02a-460b-8973-4073eac1f919 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:33.608576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9f398-f02a-460b-8973-4073eac1f919 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:33.611780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa9f398-f02a-460b-8973-4073eac1f919 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:33.627013 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 598/1204] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:33.631005 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528563d4-5e2e-46f9-b83f-8ccfa231dbc8 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:33.703730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:33.780267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528563d4-5e2e-46f9-b83f-8ccfa231dbc8 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:33.823340 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85df8d-73ae-4438-b661-b10d7aa1a18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:33.823983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85df8d-73ae-4438-b661-b10d7aa1a18b None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:33.824262 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85df8d-73ae-4438-b661-b10d7aa1a18b None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:33.824549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85df8d-73ae-4438-b661-b10d7aa1a18b None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:33.825303 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85df8d-73ae-4438-b661-b10d7aa1a18b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:33.827031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85df8d-73ae-4438-b661-b10d7aa1a18b None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:33.851871 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 607/1205] 10.222.0.48 () {60 vars in 1132 bytes} [Mon Aug 1 15:13:33 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:13:36.151545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4cac8-a2b5-489d-8242-ff50a9787624 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:36.152070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4cac8-a2b5-489d-8242-ff50a9787624 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:36.152070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4cac8-a2b5-489d-8242-ff50a9787624 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:36.153732 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 599/1206] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:13:36.159106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ba50d-f032-4087-8250-cb582bbd1f3b None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:36.159783 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ba50d-f032-4087-8250-cb582bbd1f3b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:36.159783 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ba50d-f032-4087-8250-cb582bbd1f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:36.179586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-8b9ba50d-f032-4087-8250-cb582bbd1f3b None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:36.180856 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8b9ba50d-f032-4087-8250-cb582bbd1f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:36.250669 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 608/1207] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:13:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:13:36.275972 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4cac8-a2b5-489d-8242-ff50a9787624 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:36.396693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4cac8-a2b5-489d-8242-ff50a9787624 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:36.437533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daba0a1-1719-489f-a2f3-68a2c581c2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:36.438358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daba0a1-1719-489f-a2f3-68a2c581c2eb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:36.438643 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daba0a1-1719-489f-a2f3-68a2c581c2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:36.439063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daba0a1-1719-489f-a2f3-68a2c581c2eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:36.454362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daba0a1-1719-489f-a2f3-68a2c581c2eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:13:36.459711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-9daba0a1-1719-489f-a2f3-68a2c581c2eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:13:36.459711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:36.459711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:36.459711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:36.459711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:36.459711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:36.459711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:36.459711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:36.459711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:36.459711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:36.459711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:36.459711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:36.459711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:13:36.459711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:13:36.459711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:13:36.459711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:13:36.460889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:13:36.460889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:13:36.460889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:13:36.460889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:13:36.460889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:36.460889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:36.460889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:36.460889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:36.460889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:36.460889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:36.460889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:36.460889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:36.460889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:36.460889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:36.460889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:36.460889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:36.462035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:36.462035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:36.462035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:36.462035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:36.462035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:13:36.462035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:13:36.462035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:13:36.462035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:13:36.462035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:13:36.462035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:13:36.462035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:13:36.462035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:13:36.462035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 600/1208] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:13:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:36.468144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ba50d-f032-4087-8250-cb582bbd1f3b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:36.521702 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:36.597765 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ba50d-f032-4087-8250-cb582bbd1f3b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:36.642354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26559f-d4c4-4f2f-a113-74e5882b7de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:36.643171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26559f-d4c4-4f2f-a113-74e5882b7de4 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:36.643509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26559f-d4c4-4f2f-a113-74e5882b7de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:36.643920 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26559f-d4c4-4f2f-a113-74e5882b7de4 None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:36.644668 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26559f-d4c4-4f2f-a113-74e5882b7de4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:36.644998 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26559f-d4c4-4f2f-a113-74e5882b7de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:36.646476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26559f-d4c4-4f2f-a113-74e5882b7de4 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:36.656156 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 609/1209] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:13:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:13:36.662490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daba0a1-1719-489f-a2f3-68a2c581c2eb None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:36.724740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:36.780296 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daba0a1-1719-489f-a2f3-68a2c581c2eb None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:36.825097 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be71ef-085d-4fd1-a3ae-0e41ef1369a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:36.825735 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be71ef-085d-4fd1-a3ae-0e41ef1369a6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:36.826111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be71ef-085d-4fd1-a3ae-0e41ef1369a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:36.826378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be71ef-085d-4fd1-a3ae-0e41ef1369a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:36.839180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be71ef-085d-4fd1-a3ae-0e41ef1369a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:36.840965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be71ef-085d-4fd1-a3ae-0e41ef1369a6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:36.860991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-93be71ef-085d-4fd1-a3ae-0e41ef1369a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:36.860991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:36.860991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:36.860991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:36.860991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:36.860991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:36.860991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:36.860991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:36.860991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:36.860991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:36.860991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:36.860991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:36.860991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:13:36.860991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:13:36.860991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:36.862180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:36.863239 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:13:36.863239 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:13:36.863239 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:36.863239 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:13:36.869971 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 601/1210] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:36.872094 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26559f-d4c4-4f2f-a113-74e5882b7de4 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:37.006851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:37.010396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26559f-d4c4-4f2f-a113-74e5882b7de4 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:37.057606 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85e828-68fa-4f1b-a0fa-466560914b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:37.058536 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85e828-68fa-4f1b-a0fa-466560914b37 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:37.059023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85e828-68fa-4f1b-a0fa-466560914b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:37.059375 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85e828-68fa-4f1b-a0fa-466560914b37 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:37.060177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85e828-68fa-4f1b-a0fa-466560914b37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:37.060624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85e828-68fa-4f1b-a0fa-466560914b37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:37.062393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85e828-68fa-4f1b-a0fa-466560914b37 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:37.078068 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 610/1211] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:37.087397 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be71ef-085d-4fd1-a3ae-0e41ef1369a6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:37.218892 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be71ef-085d-4fd1-a3ae-0e41ef1369a6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:37.220831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:37.261417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2761d-00e7-48ad-9f6a-7f38f6aa493d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:37.261612 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2761d-00e7-48ad-9f6a-7f38f6aa493d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:37.261890 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2761d-00e7-48ad-9f6a-7f38f6aa493d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:37.262152 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2761d-00e7-48ad-9f6a-7f38f6aa493d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:37.271860 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2761d-00e7-48ad-9f6a-7f38f6aa493d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:37.273487 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2761d-00e7-48ad-9f6a-7f38f6aa493d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:37.286607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-e8d2761d-00e7-48ad-9f6a-7f38f6aa493d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:13:37.286607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:37.286607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:37.286607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:37.286607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:37.286607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:37.286607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:37.286607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:37.286607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:37.286607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:37.286607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:37.286607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:37.286607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:13:37.286607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:13:37.286607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:13:37.286607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:37.287666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:37.288641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:37.288641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:37.288641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:13:37.288641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:13:37.288641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:13:37.288641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:13:37.288641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:13:37.288641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:13:37.288641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:13:37.288641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:13:37.288641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 602/1212] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:37.294053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85e828-68fa-4f1b-a0fa-466560914b37 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:37.422115 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85e828-68fa-4f1b-a0fa-466560914b37 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:37.463817 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edc3cf-5374-4e2e-9afd-cf238540929e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:37.464466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edc3cf-5374-4e2e-9afd-cf238540929e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:37.464780 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edc3cf-5374-4e2e-9afd-cf238540929e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:37.465054 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edc3cf-5374-4e2e-9afd-cf238540929e None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:37.465986 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edc3cf-5374-4e2e-9afd-cf238540929e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:37.466260 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edc3cf-5374-4e2e-9afd-cf238540929e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:37.467752 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edc3cf-5374-4e2e-9afd-cf238540929e None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:37.493240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 611/1213] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 15:13:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:37.495017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2761d-00e7-48ad-9f6a-7f38f6aa493d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:37.618469 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2761d-00e7-48ad-9f6a-7f38f6aa493d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:37.662065 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75396b86-45f1-4413-9f8c-ff16bb9a3d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:37.662704 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75396b86-45f1-4413-9f8c-ff16bb9a3d9d None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:37.663017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75396b86-45f1-4413-9f8c-ff16bb9a3d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:37.663279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75396b86-45f1-4413-9f8c-ff16bb9a3d9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:37.696399 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:37.744814 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75396b86-45f1-4413-9f8c-ff16bb9a3d9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:37.746920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75396b86-45f1-4413-9f8c-ff16bb9a3d9d None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:37.877648 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 603/1214] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:13:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:13:37.879244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edc3cf-5374-4e2e-9afd-cf238540929e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:37.901720 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:37.998756 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edc3cf-5374-4e2e-9afd-cf238540929e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:38.043760 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f2319-07b5-4e16-885a-d67384fe6e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:38.044477 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f2319-07b5-4e16-885a-d67384fe6e91 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:38.044740 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f2319-07b5-4e16-885a-d67384fe6e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:38.045066 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f2319-07b5-4e16-885a-d67384fe6e91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:38.057704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f2319-07b5-4e16-885a-d67384fe6e91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:38.059629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f2319-07b5-4e16-885a-d67384fe6e91 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:38.077935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-aa0f2319-07b5-4e16-885a-d67384fe6e91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:38.077935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:38.077935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:38.077935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:38.077935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:38.077935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:38.077935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:38.077935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:38.077935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:38.077935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:38.077935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:38.077935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:38.077935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:13:38.077935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:13:38.077935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:38.078987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:38.080048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:13:38.080048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:13:38.080048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:38.080048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:13:38.080048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 612/1215] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:38.086263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75396b86-45f1-4413-9f8c-ff16bb9a3d9d None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:38.162088 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:38.207992 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75396b86-45f1-4413-9f8c-ff16bb9a3d9d None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:38.249225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc27ef-79c3-48f5-8b44-b909f30c112f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:38.249975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc27ef-79c3-48f5-8b44-b909f30c112f None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:38.250236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc27ef-79c3-48f5-8b44-b909f30c112f None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:38.250453 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc27ef-79c3-48f5-8b44-b909f30c112f None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:38.251210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc27ef-79c3-48f5-8b44-b909f30c112f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:38.251538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc27ef-79c3-48f5-8b44-b909f30c112f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:38.253153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc27ef-79c3-48f5-8b44-b909f30c112f None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:38.267455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 604/1216] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:38.274323 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f2319-07b5-4e16-885a-d67384fe6e91 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:38.379578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:38.419713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f2319-07b5-4e16-885a-d67384fe6e91 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:38.457090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e968118-2e8a-48e3-b294-0476475784ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:38.458167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e968118-2e8a-48e3-b294-0476475784ca None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:38.458560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e968118-2e8a-48e3-b294-0476475784ca None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:38.458973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e968118-2e8a-48e3-b294-0476475784ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:38.466747 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e968118-2e8a-48e3-b294-0476475784ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:38.468342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e968118-2e8a-48e3-b294-0476475784ca None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:38.480694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-4e968118-2e8a-48e3-b294-0476475784ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:13:38.480694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:38.480694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:38.480694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:38.480694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:38.480694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:38.480694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:38.480694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:38.480694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:38.480694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:38.480694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:38.480694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:38.480694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:13:38.480694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:13:38.480694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:38.488629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:38.489901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:13:38.489901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:13:38.489901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:13:38.489901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:13:38.489901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:13:38.489901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:13:38.489901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:13:38.489901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:13:38.489901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 613/1217] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:13:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:13:38.491962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc27ef-79c3-48f5-8b44-b909f30c112f None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:38.621572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc27ef-79c3-48f5-8b44-b909f30c112f None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:38.649914 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:38.662354 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e589056-18c3-4417-9a99-411fbc729c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:38.663101 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e589056-18c3-4417-9a99-411fbc729c20 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:38.663444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e589056-18c3-4417-9a99-411fbc729c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:38.663782 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e589056-18c3-4417-9a99-411fbc729c20 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:38.664607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e589056-18c3-4417-9a99-411fbc729c20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:38.664979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e589056-18c3-4417-9a99-411fbc729c20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:38.666562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e589056-18c3-4417-9a99-411fbc729c20 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:38.681019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 605/1218] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:13:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:38.688673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e968118-2e8a-48e3-b294-0476475784ca None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:38.812002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e968118-2e8a-48e3-b294-0476475784ca None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:38.852445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:38.860394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc99039-28ca-44e3-b3a6-1ea81f8ae151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:38.862122 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc99039-28ca-44e3-b3a6-1ea81f8ae151 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:38.862122 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc99039-28ca-44e3-b3a6-1ea81f8ae151 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:38.862122 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc99039-28ca-44e3-b3a6-1ea81f8ae151 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:38.862754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc99039-28ca-44e3-b3a6-1ea81f8ae151 None admin] RBAC: Adding query filter params (role.id=624a5f7ed1cf440cb177291e89bb11ff, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=098d8d7e35804ccc931ddfba07f688a5) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:38.863070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc99039-28ca-44e3-b3a6-1ea81f8ae151 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:38.866668 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc99039-28ca-44e3-b3a6-1ea81f8ae151 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:38.871674 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 614/1219] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:13:38 2022] GET /identity/v3/role_assignments?user.id=098d8d7e35804ccc931ddfba07f688a5&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=624a5f7ed1cf440cb177291e89bb11ff => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:41.012796 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257a39d-c629-45a4-b55a-cc3151685a12 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:41.013357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257a39d-c629-45a4-b55a-cc3151685a12 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:41.013357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257a39d-c629-45a4-b55a-cc3151685a12 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:41.014742 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 606/1220] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:13:41.020582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5f794-78cc-4952-9055-21835243eb58 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:41.021013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5f794-78cc-4952-9055-21835243eb58 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:41.021187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5f794-78cc-4952-9055-21835243eb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:41.040812 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-54e5f794-78cc-4952-9055-21835243eb58 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:41.042129 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-54e5f794-78cc-4952-9055-21835243eb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:41.124828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 615/1221] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:13:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:13:41.152688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257a39d-c629-45a4-b55a-cc3151685a12 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:41.283090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257a39d-c629-45a4-b55a-cc3151685a12 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:41.325155 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7989d-ba1e-4a85-b37d-22271f7473d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:41.325840 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7989d-ba1e-4a85-b37d-22271f7473d7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:41.326136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7989d-ba1e-4a85-b37d-22271f7473d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:41.326426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7989d-ba1e-4a85-b37d-22271f7473d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:41.338536 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7989d-ba1e-4a85-b37d-22271f7473d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:13:41.343803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-f1e7989d-ba1e-4a85-b37d-22271f7473d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:13:41.343803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:41.343803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:41.343803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:41.343803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:41.343803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:41.343803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:41.343803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:41.343803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:41.343803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:41.343803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:41.343803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:41.343803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:13:41.343803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:13:41.343803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:13:41.343803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:13:41.344888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:13:41.344888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:13:41.344888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:13:41.344888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:13:41.344888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:41.344888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:41.344888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:41.344888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:41.344888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:41.344888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:41.344888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:41.344888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:41.344888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:41.344888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:41.344888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:41.344888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:41.345856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:41.345856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:41.345856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:41.345856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:41.345856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:13:41.345856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:13:41.345856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:13:41.345856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:13:41.345856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:13:41.345856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:13:41.345856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:13:41.345856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:13:41.345856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 607/1222] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:13:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:41.351471 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5f794-78cc-4952-9055-21835243eb58 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:41.460779 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5f794-78cc-4952-9055-21835243eb58 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:41.487161 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:41.499522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2adbb-c99d-460c-b2c2-f898b0cd34cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:41.500228 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2adbb-c99d-460c-b2c2-f898b0cd34cf None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:41.500549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2adbb-c99d-460c-b2c2-f898b0cd34cf None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:41.500838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2adbb-c99d-460c-b2c2-f898b0cd34cf None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:41.501608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2adbb-c99d-460c-b2c2-f898b0cd34cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:41.501987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2adbb-c99d-460c-b2c2-f898b0cd34cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:41.503528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2adbb-c99d-460c-b2c2-f898b0cd34cf None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:41.513260 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 616/1223] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:13:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:41.519584 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7989d-ba1e-4a85-b37d-22271f7473d7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:41.574765 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:41.656176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7989d-ba1e-4a85-b37d-22271f7473d7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:41.694559 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c88fe-b441-4e57-9eff-29dd4be5a310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:41.695231 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c88fe-b441-4e57-9eff-29dd4be5a310 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:41.695512 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c88fe-b441-4e57-9eff-29dd4be5a310 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:41.695780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c88fe-b441-4e57-9eff-29dd4be5a310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:41.706706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c88fe-b441-4e57-9eff-29dd4be5a310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:41.708437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c88fe-b441-4e57-9eff-29dd4be5a310 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:41.724323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-714c88fe-b441-4e57-9eff-29dd4be5a310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:41.724323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:41.724323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:41.724323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:41.724323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:41.724323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:41.724323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:41.724323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:41.724323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:41.724323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:41.724323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:41.724323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:41.724323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:13:41.724323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:13:41.724323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:41.726147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:41.727200 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:13:41.727200 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:13:41.727200 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:41.727200 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:13:41.727200 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 608/1224] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:13:41.732094 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2adbb-c99d-460c-b2c2-f898b0cd34cf None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:41.846336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2adbb-c99d-460c-b2c2-f898b0cd34cf None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:41.887658 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8f18f-9123-45a4-9a20-3c6bfc039dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:41.888323 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8f18f-9123-45a4-9a20-3c6bfc039dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:41.888634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8f18f-9123-45a4-9a20-3c6bfc039dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:41.889017 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8f18f-9123-45a4-9a20-3c6bfc039dc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:41.889862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8f18f-9123-45a4-9a20-3c6bfc039dc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:41.890229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8f18f-9123-45a4-9a20-3c6bfc039dc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:41.892066 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8f18f-9123-45a4-9a20-3c6bfc039dc4 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:41.904786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 617/1225] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:41.910684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c88fe-b441-4e57-9eff-29dd4be5a310 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:42.014911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:42.022738 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c88fe-b441-4e57-9eff-29dd4be5a310 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:42.064232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d87d03-0d37-4de1-bc7c-b50a82382550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:42.065135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d87d03-0d37-4de1-bc7c-b50a82382550 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:42.065476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d87d03-0d37-4de1-bc7c-b50a82382550 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:42.065746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d87d03-0d37-4de1-bc7c-b50a82382550 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:42.074946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d87d03-0d37-4de1-bc7c-b50a82382550 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:42.076512 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d87d03-0d37-4de1-bc7c-b50a82382550 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:42.088927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-09d87d03-0d37-4de1-bc7c-b50a82382550 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:13:42.088927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:42.088927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:42.088927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:42.088927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:42.088927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:42.088927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:42.088927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:42.088927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:42.088927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:42.088927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:42.088927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:42.088927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:13:42.088927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:13:42.088927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:13:42.088927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:42.090032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:42.091040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:42.091040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:42.091040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:13:42.091040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:13:42.091040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:13:42.091040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:13:42.091040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:13:42.091040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:13:42.091040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:13:42.091040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:13:42.091040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 609/1226] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:42.096160 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8f18f-9123-45a4-9a20-3c6bfc039dc4 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:42.145768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:42.206186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8f18f-9123-45a4-9a20-3c6bfc039dc4 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:42.242735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16082a24-5e53-4a5d-af68-47ceeb2603b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:42.243364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16082a24-5e53-4a5d-af68-47ceeb2603b4 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:42.243644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16082a24-5e53-4a5d-af68-47ceeb2603b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:42.243918 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16082a24-5e53-4a5d-af68-47ceeb2603b4 None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:42.244734 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16082a24-5e53-4a5d-af68-47ceeb2603b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:42.245041 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16082a24-5e53-4a5d-af68-47ceeb2603b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:42.246515 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16082a24-5e53-4a5d-af68-47ceeb2603b4 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:42.278691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 618/1227] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 15:13:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:42.279985 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d87d03-0d37-4de1-bc7c-b50a82382550 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:42.395389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d87d03-0d37-4de1-bc7c-b50a82382550 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:42.432231 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7cb28-6b33-4a46-966f-d7fb701ab77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:42.432876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7cb28-6b33-4a46-966f-d7fb701ab77c None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:42.433153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7cb28-6b33-4a46-966f-d7fb701ab77c None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:42.433425 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7cb28-6b33-4a46-966f-d7fb701ab77c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:42.444718 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7cb28-6b33-4a46-966f-d7fb701ab77c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:42.446218 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7cb28-6b33-4a46-966f-d7fb701ab77c None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:42.462240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-54b7cb28-6b33-4a46-966f-d7fb701ab77c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:42.462240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:42.462240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:42.462240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:42.462240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:42.462240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:42.462240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:42.462240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:42.462240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:42.462240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:42.462240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:42.462240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:42.462240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:13:42.462240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:13:42.462240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:42.463533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:42.465127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:13:42.465127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:13:42.465127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:42.465127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:13:42.465127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 610/1228] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:42.469703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16082a24-5e53-4a5d-af68-47ceeb2603b4 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:42.588119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16082a24-5e53-4a5d-af68-47ceeb2603b4 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:42.590660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:42.628304 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32c418-b41b-44d6-a77f-9f678f6b3446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:42.628926 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32c418-b41b-44d6-a77f-9f678f6b3446 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:42.629308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32c418-b41b-44d6-a77f-9f678f6b3446 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:42.629561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32c418-b41b-44d6-a77f-9f678f6b3446 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:42.630358 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32c418-b41b-44d6-a77f-9f678f6b3446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:42.630607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32c418-b41b-44d6-a77f-9f678f6b3446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:42.632138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32c418-b41b-44d6-a77f-9f678f6b3446 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:42.644443 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 619/1229] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:13:42.650485 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7cb28-6b33-4a46-966f-d7fb701ab77c None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:42.660139 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:42.759890 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7cb28-6b33-4a46-966f-d7fb701ab77c None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:42.800241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14557b2-8a45-41c8-bd56-226f9559a93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:42.800891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14557b2-8a45-41c8-bd56-226f9559a93a None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:42.801164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14557b2-8a45-41c8-bd56-226f9559a93a None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:42.801432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14557b2-8a45-41c8-bd56-226f9559a93a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:42.808914 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14557b2-8a45-41c8-bd56-226f9559a93a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:42.810418 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14557b2-8a45-41c8-bd56-226f9559a93a None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:42.824404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-b14557b2-8a45-41c8-bd56-226f9559a93a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:13:42.824404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:42.824404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:42.824404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:42.824404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:42.824404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:42.824404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:42.824404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:42.824404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:42.824404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:42.824404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:42.824404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:42.824404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:13:42.824404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:13:42.824404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:42.825616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:42.826790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:13:42.826790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:13:42.826790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:13:42.826790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:13:42.826790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:13:42.826790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:13:42.826790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:13:42.826790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:13:42.829880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 611/1230] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:13:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:42.832940 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32c418-b41b-44d6-a77f-9f678f6b3446 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:42.963988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32c418-b41b-44d6-a77f-9f678f6b3446 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:43.000873 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f00f9-5106-4bc7-b696-673514e4091d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:43.001559 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f00f9-5106-4bc7-b696-673514e4091d None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:43.001955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f00f9-5106-4bc7-b696-673514e4091d None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:43.002316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f00f9-5106-4bc7-b696-673514e4091d None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:43.003243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f00f9-5106-4bc7-b696-673514e4091d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:43.003672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f00f9-5106-4bc7-b696-673514e4091d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:43.005380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f00f9-5106-4bc7-b696-673514e4091d None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:43.019963 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 620/1231] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:13:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:13:43.026075 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14557b2-8a45-41c8-bd56-226f9559a93a None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:43.095731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:43.137152 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14557b2-8a45-41c8-bd56-226f9559a93a None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:43.175953 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056b8d1-7b86-4cd6-a53f-238b6793f4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:43.176636 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056b8d1-7b86-4cd6-a53f-238b6793f4c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:43.176914 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056b8d1-7b86-4cd6-a53f-238b6793f4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:43.177187 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056b8d1-7b86-4cd6-a53f-238b6793f4c6 None admin] PATH_INFO: `/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/098d8d7e35804ccc931ddfba07f688a5/roles/624a5f7ed1cf440cb177291e89bb11ff` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:43.193693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056b8d1-7b86-4cd6-a53f-238b6793f4c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=218a9ee5cb2842248b6995d2562bb74b, user_id=098d8d7e35804ccc931ddfba07f688a5, role_id=624a5f7ed1cf440cb177291e89bb11ff)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:43.195595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056b8d1-7b86-4cd6-a53f-238b6793f4c6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:43.210462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 612/1232] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:13:43 2022] PUT /identity/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/098d8d7e35804ccc931ddfba07f688a5/roles/624a5f7ed1cf440cb177291e89bb11ff => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:13:45.216956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1342bd-cfe3-49a2-8506-2a4e733f7fbd None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:45.217675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1342bd-cfe3-49a2-8506-2a4e733f7fbd None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:45.217675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1342bd-cfe3-49a2-8506-2a4e733f7fbd None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:45.219535 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 621/1233] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:13:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:13:45.225571 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344ebf2-e893-43ee-902b-ac4b1b90d93e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:45.226204 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344ebf2-e893-43ee-902b-ac4b1b90d93e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:45.226204 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344ebf2-e893-43ee-902b-ac4b1b90d93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:45.247519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-9344ebf2-e893-43ee-902b-ac4b1b90d93e None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:45.248740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9344ebf2-e893-43ee-902b-ac4b1b90d93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:45.322878 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 613/1234] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:13:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:13:45.354703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1342bd-cfe3-49a2-8506-2a4e733f7fbd None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:45.384549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:45.482968 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1342bd-cfe3-49a2-8506-2a4e733f7fbd None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:45.525697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c119a5-0ea4-4d61-883f-83d970b6644e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:45.526395 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c119a5-0ea4-4d61-883f-83d970b6644e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:45.526670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c119a5-0ea4-4d61-883f-83d970b6644e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:45.526938 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c119a5-0ea4-4d61-883f-83d970b6644e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:45.538013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c119a5-0ea4-4d61-883f-83d970b6644e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:13:45.542906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-14c119a5-0ea4-4d61-883f-83d970b6644e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:13:45.542906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:45.542906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:45.542906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:45.542906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:45.542906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:45.542906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:45.542906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:45.542906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:45.542906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:45.542906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:45.542906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:45.542906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:13:45.542906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:13:45.542906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:13:45.542906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:13:45.543969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:13:45.543969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:13:45.543969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:13:45.543969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:13:45.543969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:45.543969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:45.543969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:45.543969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:45.543969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:45.543969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:45.543969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:45.543969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:45.543969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:45.543969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:45.543969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:45.543969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:45.544807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:45.544807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:45.544807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:45.544807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:45.544807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:13:45.544807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:13:45.544807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:13:45.544807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:13:45.544807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:13:45.544807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:13:45.544807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:13:45.544807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:13:45.544807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 622/1235] 10.222.0.48 () {56 vars in 1102 bytes} [Mon Aug 1 15:13:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:45.550595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344ebf2-e893-43ee-902b-ac4b1b90d93e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:45.607062 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:45.678240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344ebf2-e893-43ee-902b-ac4b1b90d93e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:45.723027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6cbcc-e9e3-4104-aa5b-2cbc689afe6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:45.723926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6cbcc-e9e3-4104-aa5b-2cbc689afe6b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:45.724290 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6cbcc-e9e3-4104-aa5b-2cbc689afe6b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:45.724588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb6cbcc-e9e3-4104-aa5b-2cbc689afe6b None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:45.725428 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6cbcc-e9e3-4104-aa5b-2cbc689afe6b None admin] RBAC: Adding query filter params (name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:45.725868 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6cbcc-e9e3-4104-aa5b-2cbc689afe6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:45.727498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb6cbcc-e9e3-4104-aa5b-2cbc689afe6b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:45.738148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 614/1236] 10.222.0.48 () {56 vars in 1103 bytes} [Mon Aug 1 15:13:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:45.744084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c119a5-0ea4-4d61-883f-83d970b6644e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:45.842639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:45.861634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c119a5-0ea4-4d61-883f-83d970b6644e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:45.898771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52432667-f217-4089-aa2f-8d67aa5361bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:45.899417 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52432667-f217-4089-aa2f-8d67aa5361bf None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:45.899692 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52432667-f217-4089-aa2f-8d67aa5361bf None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:45.899960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52432667-f217-4089-aa2f-8d67aa5361bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:45.911963 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52432667-f217-4089-aa2f-8d67aa5361bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:45.913405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52432667-f217-4089-aa2f-8d67aa5361bf None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:45.932612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-52432667-f217-4089-aa2f-8d67aa5361bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:45.932612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:45.932612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:45.932612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:45.932612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:45.932612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:45.932612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:45.932612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:45.932612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:45.932612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:45.932612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:45.932612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:45.932612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:13:45.932612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:13:45.932612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:45.933639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:45.939944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:13:45.939944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:13:45.939944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:45.939944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:13:45.939944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 623/1237] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:45.940371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6cbcc-e9e3-4104-aa5b-2cbc689afe6b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:46.055629 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb6cbcc-e9e3-4104-aa5b-2cbc689afe6b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:46.066746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:46.098133 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde8fad-9cec-41c9-a555-96706eb318a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:46.098754 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde8fad-9cec-41c9-a555-96706eb318a9 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:46.099031 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde8fad-9cec-41c9-a555-96706eb318a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:46.099302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde8fad-9cec-41c9-a555-96706eb318a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:46.100028 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde8fad-9cec-41c9-a555-96706eb318a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:46.100329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde8fad-9cec-41c9-a555-96706eb318a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:46.101718 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde8fad-9cec-41c9-a555-96706eb318a9 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:46.116569 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 615/1238] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:46.123024 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52432667-f217-4089-aa2f-8d67aa5361bf None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:46.239417 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52432667-f217-4089-aa2f-8d67aa5361bf None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:46.284228 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470afda-9d67-4e7c-bab9-d90145606c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:46.284853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470afda-9d67-4e7c-bab9-d90145606c88 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:46.285128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470afda-9d67-4e7c-bab9-d90145606c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:46.285389 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470afda-9d67-4e7c-bab9-d90145606c88 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:46.288620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:46.293867 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0470afda-9d67-4e7c-bab9-d90145606c88 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:46.295596 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0470afda-9d67-4e7c-bab9-d90145606c88 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:46.309185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-0470afda-9d67-4e7c-bab9-d90145606c88 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:13:46.309185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:46.309185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:46.309185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:46.309185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:46.309185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:46.309185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:46.309185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:46.309185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:46.309185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:46.309185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:46.309185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:46.309185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:13:46.309185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:13:46.309185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:13:46.309185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:46.310352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:46.311310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:46.311310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:46.311310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:13:46.311310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:13:46.311310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:13:46.311310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:13:46.311310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:13:46.311310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:13:46.311310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:13:46.311310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:13:46.311310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 624/1239] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:46.317610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde8fad-9cec-41c9-a555-96706eb318a9 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:46.431777 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde8fad-9cec-41c9-a555-96706eb318a9 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:46.468788 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d5965-d2f4-442b-bf28-496f9d9a4fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:46.469856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d5965-d2f4-442b-bf28-496f9d9a4fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:46.470108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d5965-d2f4-442b-bf28-496f9d9a4fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:46.470601 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d5965-d2f4-442b-bf28-496f9d9a4fd4 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:46.471553 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d5965-d2f4-442b-bf28-496f9d9a4fd4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:46.471940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d5965-d2f4-442b-bf28-496f9d9a4fd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:46.473494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d5965-d2f4-442b-bf28-496f9d9a4fd4 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:46.493688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 616/1240] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 15:13:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:46.499800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470afda-9d67-4e7c-bab9-d90145606c88 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:46.611498 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470afda-9d67-4e7c-bab9-d90145606c88 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:46.650870 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235aeac-f982-4735-ae76-f998cafb8a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:46.651420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235aeac-f982-4735-ae76-f998cafb8a1e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:46.651701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235aeac-f982-4735-ae76-f998cafb8a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:46.651998 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235aeac-f982-4735-ae76-f998cafb8a1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:46.727350 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:46.736065 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235aeac-f982-4735-ae76-f998cafb8a1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:46.737843 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235aeac-f982-4735-ae76-f998cafb8a1e None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:46.853878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 625/1241] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:13:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:13:46.859908 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d5965-d2f4-442b-bf28-496f9d9a4fd4 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:46.890027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:46.975874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d5965-d2f4-442b-bf28-496f9d9a4fd4 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:47.018551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5b425-1286-43bb-a7b8-506b45a13353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:47.019210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5b425-1286-43bb-a7b8-506b45a13353 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:47.019476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5b425-1286-43bb-a7b8-506b45a13353 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:47.019750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5b425-1286-43bb-a7b8-506b45a13353 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:47.031352 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5b425-1286-43bb-a7b8-506b45a13353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:47.032859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5b425-1286-43bb-a7b8-506b45a13353 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:47.048773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-a2d5b425-1286-43bb-a7b8-506b45a13353 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:47.048773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:47.048773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:47.048773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:47.048773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:47.048773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:47.048773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:47.048773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:47.048773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:47.048773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:47.048773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:47.048773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:47.048773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:13:47.048773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:13:47.048773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:47.053813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:47.055027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:13:47.055027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:13:47.055027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:47.055027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:13:47.055027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 617/1242] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:47.056676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235aeac-f982-4735-ae76-f998cafb8a1e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:47.166640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:47.174267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235aeac-f982-4735-ae76-f998cafb8a1e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:47.212219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b5099-5e5a-4b6f-ba22-4fc0774f4439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:47.212842 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b5099-5e5a-4b6f-ba22-4fc0774f4439 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:47.213119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b5099-5e5a-4b6f-ba22-4fc0774f4439 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:47.213391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b5099-5e5a-4b6f-ba22-4fc0774f4439 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:47.214129 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b5099-5e5a-4b6f-ba22-4fc0774f4439 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:47.214448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b5099-5e5a-4b6f-ba22-4fc0774f4439 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:47.215816 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b5099-5e5a-4b6f-ba22-4fc0774f4439 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:47.230776 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 626/1243] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:47.236973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5b425-1286-43bb-a7b8-506b45a13353 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:47.338814 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:47.364205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5b425-1286-43bb-a7b8-506b45a13353 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:47.408169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f087e0-1721-4271-95dd-324e61d7edc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:47.408864 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f087e0-1721-4271-95dd-324e61d7edc0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:47.409213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f087e0-1721-4271-95dd-324e61d7edc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:47.409478 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f087e0-1721-4271-95dd-324e61d7edc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:47.416529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f087e0-1721-4271-95dd-324e61d7edc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:47.418272 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f087e0-1721-4271-95dd-324e61d7edc0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:47.430392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-47f087e0-1721-4271-95dd-324e61d7edc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:13:47.430392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:47.430392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:47.430392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:47.430392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:47.430392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:47.430392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:47.430392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:47.430392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:47.430392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:47.430392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:47.430392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:47.430392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:13:47.430392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:13:47.430392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:47.431438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:47.432687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:13:47.432687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:13:47.432687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:13:47.432687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:13:47.432687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:13:47.432687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:13:47.432687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:13:47.432687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:13:47.432687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 618/1244] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:13:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:47.438119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b5099-5e5a-4b6f-ba22-4fc0774f4439 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:47.551441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b5099-5e5a-4b6f-ba22-4fc0774f4439 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:47.592886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1c940-7f44-47a2-ac3f-66ebc9b15765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:47.593523 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1c940-7f44-47a2-ac3f-66ebc9b15765 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:47.593773 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1c940-7f44-47a2-ac3f-66ebc9b15765 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:47.594109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1c940-7f44-47a2-ac3f-66ebc9b15765 None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:47.594833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1c940-7f44-47a2-ac3f-66ebc9b15765 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:47.595134 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1c940-7f44-47a2-ac3f-66ebc9b15765 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:47.596620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1c940-7f44-47a2-ac3f-66ebc9b15765 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:47.610506 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 627/1245] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:13:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:13:47.616824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f087e0-1721-4271-95dd-324e61d7edc0 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:47.736460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f087e0-1721-4271-95dd-324e61d7edc0 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:47.772490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78605aa6-e035-4774-b413-aa426b490812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:47.773290 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78605aa6-e035-4774-b413-aa426b490812 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:47.773653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78605aa6-e035-4774-b413-aa426b490812 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:47.773996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78605aa6-e035-4774-b413-aa426b490812 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:47.774939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78605aa6-e035-4774-b413-aa426b490812 None admin] RBAC: Adding query filter params (role.id=624a5f7ed1cf440cb177291e89bb11ff, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=098d8d7e35804ccc931ddfba07f688a5) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:47.775321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78605aa6-e035-4774-b413-aa426b490812 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:47.776780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78605aa6-e035-4774-b413-aa426b490812 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:47.797816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 619/1246] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:13:47 2022] GET /identity/v3/role_assignments?user.id=098d8d7e35804ccc931ddfba07f688a5&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=624a5f7ed1cf440cb177291e89bb11ff => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:13:49.981574 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3f38e-5412-44c5-b4c2-4c8dd27a2709 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:49.982142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3f38e-5412-44c5-b4c2-4c8dd27a2709 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:49.982237 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3f38e-5412-44c5-b4c2-4c8dd27a2709 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:49.983937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 628/1247] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:13:49.989615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c765b-c5bf-4170-82ef-74ba5c2b902a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:49.990131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c765b-c5bf-4170-82ef-74ba5c2b902a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:49.990274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c765b-c5bf-4170-82ef-74ba5c2b902a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:49.997007 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:50.009974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-aa7c765b-c5bf-4170-82ef-74ba5c2b902a None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:50.011338 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-aa7c765b-c5bf-4170-82ef-74ba5c2b902a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:50.086558 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 620/1248] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:13:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:13:50.110287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3f38e-5412-44c5-b4c2-4c8dd27a2709 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:50.131663 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:50.223970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3f38e-5412-44c5-b4c2-4c8dd27a2709 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:50.259142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ef302-03c3-4cef-81d2-b25fd694a564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:50.259799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ef302-03c3-4cef-81d2-b25fd694a564 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:50.260082 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ef302-03c3-4cef-81d2-b25fd694a564 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:50.260391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ef302-03c3-4cef-81d2-b25fd694a564 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:50.273118 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ef302-03c3-4cef-81d2-b25fd694a564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:13:50.278369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-e69ef302-03c3-4cef-81d2-b25fd694a564 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:13:50.278369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:50.278369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:50.278369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:50.278369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:50.278369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:50.278369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:50.278369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:50.278369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:50.278369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:50.278369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:50.278369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:50.278369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:13:50.278369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:13:50.278369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:13:50.278369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:13:50.279571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:13:50.279571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:13:50.279571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:13:50.279571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:13:50.279571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:50.279571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:50.279571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:50.279571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:50.279571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:50.279571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:50.279571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:50.279571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:50.279571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:50.279571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:50.279571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:50.279571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:50.280464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:50.280464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:50.280464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:50.280464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:50.280464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:13:50.280464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:13:50.280464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:13:50.280464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:13:50.280464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:13:50.280464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:13:50.280464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:13:50.280464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:13:50.280464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 629/1249] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:13:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:50.286049 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c765b-c5bf-4170-82ef-74ba5c2b902a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:50.408997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c765b-c5bf-4170-82ef-74ba5c2b902a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:50.444073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:50.450184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1e9cd-59eb-4e2b-8eee-79354e257205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:50.450911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1e9cd-59eb-4e2b-8eee-79354e257205 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:50.451297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1e9cd-59eb-4e2b-8eee-79354e257205 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:50.451694 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e1e9cd-59eb-4e2b-8eee-79354e257205 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:50.452634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1e9cd-59eb-4e2b-8eee-79354e257205 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:50.453025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1e9cd-59eb-4e2b-8eee-79354e257205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:50.454695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e1e9cd-59eb-4e2b-8eee-79354e257205 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:50.464221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 621/1250] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:13:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:50.470145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ef302-03c3-4cef-81d2-b25fd694a564 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:50.560538 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:50.586072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ef302-03c3-4cef-81d2-b25fd694a564 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:50.627018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451a681-715b-46cf-8c6b-ca3b2b1d397e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:50.628079 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451a681-715b-46cf-8c6b-ca3b2b1d397e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:50.628567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451a681-715b-46cf-8c6b-ca3b2b1d397e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:50.629092 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451a681-715b-46cf-8c6b-ca3b2b1d397e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:50.642358 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451a681-715b-46cf-8c6b-ca3b2b1d397e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:50.644032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451a681-715b-46cf-8c6b-ca3b2b1d397e None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:50.661608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-5451a681-715b-46cf-8c6b-ca3b2b1d397e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:50.661608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:50.661608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:50.661608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:50.661608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:50.661608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:50.661608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:50.661608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:50.661608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:50.661608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:50.661608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:50.661608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:50.661608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:13:50.661608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:13:50.661608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:50.662789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:50.663852 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:13:50.663852 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:13:50.663852 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:50.663852 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:13:50.663852 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 630/1251] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:50.669674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1e9cd-59eb-4e2b-8eee-79354e257205 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:50.791262 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e1e9cd-59eb-4e2b-8eee-79354e257205 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:50.828643 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8a27b-d607-46b6-bed8-ed9e1b39e731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:50.829323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8a27b-d607-46b6-bed8-ed9e1b39e731 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:50.829651 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8a27b-d607-46b6-bed8-ed9e1b39e731 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:50.830037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8a27b-d607-46b6-bed8-ed9e1b39e731 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:50.830761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8a27b-d607-46b6-bed8-ed9e1b39e731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:50.831094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8a27b-d607-46b6-bed8-ed9e1b39e731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:50.832551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8a27b-d607-46b6-bed8-ed9e1b39e731 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:50.849742 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 622/1252] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:50.851725 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451a681-715b-46cf-8c6b-ca3b2b1d397e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:50.967471 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451a681-715b-46cf-8c6b-ca3b2b1d397e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:50.998052 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:51.008430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50af244-7e90-422f-a8d6-7a31a35f3255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:51.009370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50af244-7e90-422f-a8d6-7a31a35f3255 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:51.009873 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50af244-7e90-422f-a8d6-7a31a35f3255 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:51.010223 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50af244-7e90-422f-a8d6-7a31a35f3255 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:51.021311 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50af244-7e90-422f-a8d6-7a31a35f3255 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:51.022934 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50af244-7e90-422f-a8d6-7a31a35f3255 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:51.037662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-a50af244-7e90-422f-a8d6-7a31a35f3255 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:13:51.037662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:51.037662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:51.037662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:51.037662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:51.037662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:51.037662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:51.037662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:51.037662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:51.037662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:51.037662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:51.037662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:51.037662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:13:51.037662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:13:51.037662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:13:51.037662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:51.039885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:51.041380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:51.041380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:51.041380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:13:51.041380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:13:51.041380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:13:51.041380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:13:51.041380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:13:51.041380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:13:51.041380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:13:51.041380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:13:51.041380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 631/1253] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:13:51.047131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8a27b-d607-46b6-bed8-ed9e1b39e731 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:51.115830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:51.167086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8a27b-d607-46b6-bed8-ed9e1b39e731 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:51.209362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65791044-ae74-40f8-806d-84ca2ae7a935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:51.210102 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65791044-ae74-40f8-806d-84ca2ae7a935 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:51.210390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65791044-ae74-40f8-806d-84ca2ae7a935 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:51.210664 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65791044-ae74-40f8-806d-84ca2ae7a935 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:51.211672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65791044-ae74-40f8-806d-84ca2ae7a935 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:51.212031 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65791044-ae74-40f8-806d-84ca2ae7a935 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:51.213728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65791044-ae74-40f8-806d-84ca2ae7a935 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:51.239528 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 623/1254] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 15:13:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:51.240759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50af244-7e90-422f-a8d6-7a31a35f3255 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:51.367056 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50af244-7e90-422f-a8d6-7a31a35f3255 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:51.408400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76d33f-c3fc-468f-9879-e1f75371769e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:51.409179 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76d33f-c3fc-468f-9879-e1f75371769e None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:51.409476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76d33f-c3fc-468f-9879-e1f75371769e None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:51.409889 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76d33f-c3fc-468f-9879-e1f75371769e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:51.477747 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:51.493312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76d33f-c3fc-468f-9879-e1f75371769e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:51.495020 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76d33f-c3fc-468f-9879-e1f75371769e None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:51.605234 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 632/1255] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:13:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 15:13:51.611824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65791044-ae74-40f8-806d-84ca2ae7a935 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:51.726149 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65791044-ae74-40f8-806d-84ca2ae7a935 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:51.743906 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:51.768103 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb79738-589c-4f9e-a352-ef407863fe9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:51.768765 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb79738-589c-4f9e-a352-ef407863fe9b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:51.769039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb79738-589c-4f9e-a352-ef407863fe9b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:51.769332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb79738-589c-4f9e-a352-ef407863fe9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:51.782063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb79738-589c-4f9e-a352-ef407863fe9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:51.783622 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb79738-589c-4f9e-a352-ef407863fe9b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:51.799822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-5eb79738-589c-4f9e-a352-ef407863fe9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:51.799822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:51.799822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:51.799822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:51.799822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:51.799822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:51.799822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:51.799822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:51.799822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:51.799822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:51.799822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:51.799822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:51.799822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:13:51.799822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:13:51.799822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:51.801082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:51.802023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:13:51.802023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:13:51.802023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:51.802023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:13:51.802023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 624/1256] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:51.809631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76d33f-c3fc-468f-9879-e1f75371769e None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:51.914439 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:51.929456 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76d33f-c3fc-468f-9879-e1f75371769e None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:51.968628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c28383-c029-48b0-99d0-27c85e7c7ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:51.969307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c28383-c029-48b0-99d0-27c85e7c7ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:51.969630 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c28383-c029-48b0-99d0-27c85e7c7ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:51.970008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c28383-c029-48b0-99d0-27c85e7c7ca5 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:51.970829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c28383-c029-48b0-99d0-27c85e7c7ca5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:51.971204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c28383-c029-48b0-99d0-27c85e7c7ca5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:51.972966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c28383-c029-48b0-99d0-27c85e7c7ca5 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:51.986629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 633/1257] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:13:51.992891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb79738-589c-4f9e-a352-ef407863fe9b None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:52.111648 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb79738-589c-4f9e-a352-ef407863fe9b None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:52.152620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460adf5e-2f6c-497a-80c7-effcd9398c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:52.153404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460adf5e-2f6c-497a-80c7-effcd9398c00 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:52.153723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460adf5e-2f6c-497a-80c7-effcd9398c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:52.154083 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460adf5e-2f6c-497a-80c7-effcd9398c00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:52.162836 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460adf5e-2f6c-497a-80c7-effcd9398c00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:52.164325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460adf5e-2f6c-497a-80c7-effcd9398c00 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:52.178312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-460adf5e-2f6c-497a-80c7-effcd9398c00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:13:52.178312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:52.178312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:52.178312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:52.178312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:52.178312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:52.178312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:52.178312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:52.178312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:52.178312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:52.178312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:52.178312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:52.178312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:13:52.178312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:13:52.178312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:52.179518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:52.180516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:13:52.180516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:13:52.180516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:13:52.180516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:13:52.180516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:13:52.180516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:13:52.180516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:13:52.180516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:13:52.180516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 625/1258] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:13:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:52.186374 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c28383-c029-48b0-99d0-27c85e7c7ca5 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:52.301704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c28383-c029-48b0-99d0-27c85e7c7ca5 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:52.343200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d4f49-291d-42cd-a5aa-882f1ec12438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:52.343916 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d4f49-291d-42cd-a5aa-882f1ec12438 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:52.344168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d4f49-291d-42cd-a5aa-882f1ec12438 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:52.344518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d4f49-291d-42cd-a5aa-882f1ec12438 None admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:52.345506 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d4f49-291d-42cd-a5aa-882f1ec12438 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:52.345902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d4f49-291d-42cd-a5aa-882f1ec12438 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:52.347624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d4f49-291d-42cd-a5aa-882f1ec12438 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:52.363434 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 634/1259] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:13:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:13:52.370055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460adf5e-2f6c-497a-80c7-effcd9398c00 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:52.403141 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:52.498466 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460adf5e-2f6c-497a-80c7-effcd9398c00 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:52.544237 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f5780-64a5-402e-b5e3-1ed172cdc345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:52.545131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f5780-64a5-402e-b5e3-1ed172cdc345 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:52.545485 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f5780-64a5-402e-b5e3-1ed172cdc345 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:52.545902 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f5780-64a5-402e-b5e3-1ed172cdc345 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:52.547020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f5780-64a5-402e-b5e3-1ed172cdc345 None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=098d8d7e35804ccc931ddfba07f688a5) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:52.547484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f5780-64a5-402e-b5e3-1ed172cdc345 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:52.549536 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f5780-64a5-402e-b5e3-1ed172cdc345 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:52.558268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 626/1260] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:13:52 2022] GET /identity/v3/role_assignments?user.id=098d8d7e35804ccc931ddfba07f688a5&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:13:54.577085 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a6cd4-cf56-4b7d-863d-060409aca607 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:54.577674 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a6cd4-cf56-4b7d-863d-060409aca607 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:54.577674 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a6cd4-cf56-4b7d-863d-060409aca607 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:54.579030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 635/1261] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:13:54.584455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3efa3-2db2-4964-8dba-241f18ccb081 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:54.584949 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3efa3-2db2-4964-8dba-241f18ccb081 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:54.585051 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3efa3-2db2-4964-8dba-241f18ccb081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:54.604345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-80b3efa3-2db2-4964-8dba-241f18ccb081 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:54.605564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-80b3efa3-2db2-4964-8dba-241f18ccb081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:54.682615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 627/1262] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:13:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:13:54.706075 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a6cd4-cf56-4b7d-863d-060409aca607 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:54.718708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:54.815250 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a6cd4-cf56-4b7d-863d-060409aca607 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:54.853661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f66cf-f1fd-4aba-92ad-eb3d2d19dec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:54.854228 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f66cf-f1fd-4aba-92ad-eb3d2d19dec4 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:54.854499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f66cf-f1fd-4aba-92ad-eb3d2d19dec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:54.854771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f66cf-f1fd-4aba-92ad-eb3d2d19dec4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:54.865564 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f66cf-f1fd-4aba-92ad-eb3d2d19dec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:13:54.870514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-473f66cf-f1fd-4aba-92ad-eb3d2d19dec4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:13:54.870514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:54.870514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:54.870514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:54.870514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:54.870514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:54.870514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:54.870514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:54.870514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:54.870514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:54.870514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:54.870514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:54.870514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:13:54.870514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:13:54.870514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:13:54.870514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:13:54.871603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:13:54.871603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:13:54.871603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:13:54.871603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:13:54.871603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:54.871603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:54.871603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:54.871603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:54.871603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:54.871603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:54.871603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:54.871603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:54.871603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:54.871603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:54.871603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:54.871603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:54.872706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:54.872706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:54.872706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:54.872706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:54.872706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:13:54.872706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:13:54.872706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:13:54.872706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:13:54.872706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:13:54.872706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:13:54.872706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:13:54.872706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:13:54.872706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 636/1263] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:13:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:54.878044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3efa3-2db2-4964-8dba-241f18ccb081 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:54.898055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:54.997418 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3efa3-2db2-4964-8dba-241f18ccb081 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:55.034197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa51d8a-c891-4e50-b284-a2c72d2ef454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:55.034811 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa51d8a-c891-4e50-b284-a2c72d2ef454 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:55.035092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa51d8a-c891-4e50-b284-a2c72d2ef454 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:55.035373 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa51d8a-c891-4e50-b284-a2c72d2ef454 None admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:55.036093 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa51d8a-c891-4e50-b284-a2c72d2ef454 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:55.036394 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa51d8a-c891-4e50-b284-a2c72d2ef454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:55.037814 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa51d8a-c891-4e50-b284-a2c72d2ef454 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:55.046082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 628/1264] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:13:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:55.055897 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f66cf-f1fd-4aba-92ad-eb3d2d19dec4 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:55.130878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:55.169545 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f66cf-f1fd-4aba-92ad-eb3d2d19dec4 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:55.208331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a370a0-a5c1-4c4d-813b-1e4698415b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:55.209006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a370a0-a5c1-4c4d-813b-1e4698415b05 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:55.209283 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a370a0-a5c1-4c4d-813b-1e4698415b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:55.209556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a370a0-a5c1-4c4d-813b-1e4698415b05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:55.221011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a370a0-a5c1-4c4d-813b-1e4698415b05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:55.222586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a370a0-a5c1-4c4d-813b-1e4698415b05 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:55.238887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-e2a370a0-a5c1-4c4d-813b-1e4698415b05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:55.238887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:55.238887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:55.238887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:55.238887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:55.238887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:55.238887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:55.238887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:55.238887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:55.238887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:55.238887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:55.238887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:55.238887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:13:55.238887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:13:55.238887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:55.239930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:55.241111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:13:55.241111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:13:55.241111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:55.241111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:13:55.241111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 637/1265] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:55.247274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa51d8a-c891-4e50-b284-a2c72d2ef454 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:55.357378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:55.378543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa51d8a-c891-4e50-b284-a2c72d2ef454 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:55.416258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252acc79-75f7-4318-bc88-bcec6fc4dcf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:55.416787 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252acc79-75f7-4318-bc88-bcec6fc4dcf7 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:55.417058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252acc79-75f7-4318-bc88-bcec6fc4dcf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:55.417322 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252acc79-75f7-4318-bc88-bcec6fc4dcf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:55.418065 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252acc79-75f7-4318-bc88-bcec6fc4dcf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:55.418358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252acc79-75f7-4318-bc88-bcec6fc4dcf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:55.419716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252acc79-75f7-4318-bc88-bcec6fc4dcf7 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:55.431964 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 629/1266] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:55.438295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a370a0-a5c1-4c4d-813b-1e4698415b05 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:55.546834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a370a0-a5c1-4c4d-813b-1e4698415b05 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:55.569114 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:55.585197 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f5d4f-1b4c-4cef-a2da-4cabe9cb3930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:55.585876 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f5d4f-1b4c-4cef-a2da-4cabe9cb3930 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:55.586140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f5d4f-1b4c-4cef-a2da-4cabe9cb3930 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:55.586413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f5d4f-1b4c-4cef-a2da-4cabe9cb3930 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:55.594217 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f5d4f-1b4c-4cef-a2da-4cabe9cb3930 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:55.595732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f5d4f-1b4c-4cef-a2da-4cabe9cb3930 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:55.607390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-af7f5d4f-1b4c-4cef-a2da-4cabe9cb3930 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:13:55.607390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:55.607390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:55.607390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:55.607390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:55.607390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:55.607390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:55.607390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:55.607390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:55.607390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:55.607390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:55.607390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:55.607390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:13:55.607390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:13:55.607390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:13:55.607390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:55.608499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:55.609966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:55.609966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:55.609966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:13:55.609966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:13:55.609966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:13:55.609966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:13:55.609966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:13:55.609966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:13:55.609966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:13:55.609966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:13:55.609966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 638/1267] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:55.615292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252acc79-75f7-4318-bc88-bcec6fc4dcf7 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:55.724265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252acc79-75f7-4318-bc88-bcec6fc4dcf7 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:55.771891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbbd70-7c62-4404-ad4a-6a0c05fc1d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:55.772875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbbd70-7c62-4404-ad4a-6a0c05fc1d61 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:55.773180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbbd70-7c62-4404-ad4a-6a0c05fc1d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:55.773469 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbbd70-7c62-4404-ad4a-6a0c05fc1d61 None admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:55.774790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbbd70-7c62-4404-ad4a-6a0c05fc1d61 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:55.775228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbbd70-7c62-4404-ad4a-6a0c05fc1d61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:55.777527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbbd70-7c62-4404-ad4a-6a0c05fc1d61 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:55.788710 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:55.797721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 630/1268] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 15:13:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:55.803826 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f5d4f-1b4c-4cef-a2da-4cabe9cb3930 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:55.913475 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f5d4f-1b4c-4cef-a2da-4cabe9cb3930 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:55.959273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24a350-a794-4e78-9c46-e7f7fe05c5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:55.959905 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24a350-a794-4e78-9c46-e7f7fe05c5bc None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:55.960181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24a350-a794-4e78-9c46-e7f7fe05c5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:55.960446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24a350-a794-4e78-9c46-e7f7fe05c5bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:55.971552 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24a350-a794-4e78-9c46-e7f7fe05c5bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:55.972998 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24a350-a794-4e78-9c46-e7f7fe05c5bc None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:55.988138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-be24a350-a794-4e78-9c46-e7f7fe05c5bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:55.988138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:55.988138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:55.988138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:55.988138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:55.988138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:55.988138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:55.988138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:55.988138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:55.988138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:55.988138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:55.988138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:55.988138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:13:55.988138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:13:55.988138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:55.989207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:55.990665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:13:55.990665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:13:55.990665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:55.990665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:13:55.990665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 639/1269] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:55.995771 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbbd70-7c62-4404-ad4a-6a0c05fc1d61 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:56.113605 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbbd70-7c62-4404-ad4a-6a0c05fc1d61 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:56.149041 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc7322-524c-4136-b17d-8f078279d4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:56.149657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc7322-524c-4136-b17d-8f078279d4fb None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:56.149962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc7322-524c-4136-b17d-8f078279d4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:56.150235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc7322-524c-4136-b17d-8f078279d4fb None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:56.150910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc7322-524c-4136-b17d-8f078279d4fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:56.151233 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc7322-524c-4136-b17d-8f078279d4fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:56.152617 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc7322-524c-4136-b17d-8f078279d4fb None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:56.164272 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 631/1270] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:56.170187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24a350-a794-4e78-9c46-e7f7fe05c5bc None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:56.203282 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:56.277770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24a350-a794-4e78-9c46-e7f7fe05c5bc None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:56.315624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6979099-2612-4600-9e06-88a470246484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:56.316293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6979099-2612-4600-9e06-88a470246484 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:56.316640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6979099-2612-4600-9e06-88a470246484 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:56.316965 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6979099-2612-4600-9e06-88a470246484 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:56.324703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6979099-2612-4600-9e06-88a470246484 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:56.326629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6979099-2612-4600-9e06-88a470246484 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:56.342120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-b6979099-2612-4600-9e06-88a470246484 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:13:56.342120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:56.342120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:56.342120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:56.342120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:56.342120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:56.342120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:56.342120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:56.342120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:56.342120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:56.342120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:56.342120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:56.342120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:13:56.342120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:13:56.342120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:56.343243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:56.344324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:13:56.344324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:13:56.344324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:13:56.344324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:13:56.344324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:13:56.344324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:13:56.344324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:13:56.344324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:13:56.344324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 640/1271] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:13:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:56.349485 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc7322-524c-4136-b17d-8f078279d4fb None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:56.425358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:56.475900 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc7322-524c-4136-b17d-8f078279d4fb None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:56.512177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f31c9e-2567-46da-ba7d-b8f1d98b850e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:56.512791 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f31c9e-2567-46da-ba7d-b8f1d98b850e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:56.513073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f31c9e-2567-46da-ba7d-b8f1d98b850e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:56.513348 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f31c9e-2567-46da-ba7d-b8f1d98b850e None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:56.514261 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f31c9e-2567-46da-ba7d-b8f1d98b850e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:56.514261 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f31c9e-2567-46da-ba7d-b8f1d98b850e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:56.515909 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f31c9e-2567-46da-ba7d-b8f1d98b850e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:56.529503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 632/1272] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:13:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:56.537091 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6979099-2612-4600-9e06-88a470246484 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:56.639472 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:56.654827 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6979099-2612-4600-9e06-88a470246484 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:56.694038 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2502df-dd43-4a9e-b379-7d65c2dad0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:56.694709 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2502df-dd43-4a9e-b379-7d65c2dad0d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:56.694983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2502df-dd43-4a9e-b379-7d65c2dad0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:56.695265 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2502df-dd43-4a9e-b379-7d65c2dad0d2 None admin] PATH_INFO: `/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/098d8d7e35804ccc931ddfba07f688a5/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:56.697599 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2502df-dd43-4a9e-b379-7d65c2dad0d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=218a9ee5cb2842248b6995d2562bb74b, user_id=098d8d7e35804ccc931ddfba07f688a5, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:56.699087 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2502df-dd43-4a9e-b379-7d65c2dad0d2 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:56.715507 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 641/1273] 10.222.0.48 () {58 vars in 1439 bytes} [Mon Aug 1 15:13:56 2022] PUT /identity/v3/projects/218a9ee5cb2842248b6995d2562bb74b/users/098d8d7e35804ccc931ddfba07f688a5/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:13:58.743444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f48c9d-e165-49be-8894-8f4f739789ac None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:58.744009 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f48c9d-e165-49be-8894-8f4f739789ac None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:58.744009 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f48c9d-e165-49be-8894-8f4f739789ac None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:58.745355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 633/1274] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:13:58.750838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3ebd2-0724-477f-938c-1742f40bdb9e None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:58.751379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3ebd2-0724-477f-938c-1742f40bdb9e None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:58.751486 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3ebd2-0724-477f-938c-1742f40bdb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:58.769860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-01f3ebd2-0724-477f-938c-1742f40bdb9e None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:13:58.771048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-01f3ebd2-0724-477f-938c-1742f40bdb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:13:58.843860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 642/1275] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:13:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:13:58.867419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f48c9d-e165-49be-8894-8f4f739789ac None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:58.995244 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f48c9d-e165-49be-8894-8f4f739789ac None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:59.013395 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:59.033363 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc1832-055f-40fb-86d3-344b3733da91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:59.034035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc1832-055f-40fb-86d3-344b3733da91 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:59.034337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc1832-055f-40fb-86d3-344b3733da91 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:59.034607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc1832-055f-40fb-86d3-344b3733da91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:59.045398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc1832-055f-40fb-86d3-344b3733da91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:13:59.050626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-b6cc1832-055f-40fb-86d3-344b3733da91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:13:59.050626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:59.050626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:59.050626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:59.050626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:59.050626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:59.050626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:59.050626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:59.050626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:59.050626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:59.050626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:59.050626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:59.050626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:13:59.050626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:13:59.050626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:13:59.050626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:13:59.051773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:13:59.051773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:13:59.051773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:13:59.051773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:13:59.051773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:59.051773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:59.051773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:59.051773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:59.051773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:59.051773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:59.051773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:59.051773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:59.051773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:59.051773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:59.051773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:59.051773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:59.052973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:59.052973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:59.052973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:59.052973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:59.052973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:13:59.052973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:13:59.052973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:13:59.052973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:13:59.052973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:13:59.052973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:13:59.052973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:13:59.052973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:13:59.052973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 634/1276] 10.222.0.48 () {56 vars in 1096 bytes} [Mon Aug 1 15:13:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:59.061517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3ebd2-0724-477f-938c-1742f40bdb9e None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:59.149996 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:59.180014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3ebd2-0724-477f-938c-1742f40bdb9e None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:59.218747 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957353c-27b4-4242-92b5-30cc7df611cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:59.219365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957353c-27b4-4242-92b5-30cc7df611cd None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:59.219643 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957353c-27b4-4242-92b5-30cc7df611cd None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:59.219917 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957353c-27b4-4242-92b5-30cc7df611cd None admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:59.220733 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957353c-27b4-4242-92b5-30cc7df611cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:59.221040 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957353c-27b4-4242-92b5-30cc7df611cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:59.222481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957353c-27b4-4242-92b5-30cc7df611cd None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:59.231269 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 643/1277] 10.222.0.48 () {56 vars in 1099 bytes} [Mon Aug 1 15:13:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:59.237811 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc1832-055f-40fb-86d3-344b3733da91 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:59.350991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc1832-055f-40fb-86d3-344b3733da91 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:59.393497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d25ee-3a82-439e-af5e-e2936860c95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:59.394398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d25ee-3a82-439e-af5e-e2936860c95e None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:59.394804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d25ee-3a82-439e-af5e-e2936860c95e None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:59.395194 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d25ee-3a82-439e-af5e-e2936860c95e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:59.411294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d25ee-3a82-439e-af5e-e2936860c95e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:59.412761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d25ee-3a82-439e-af5e-e2936860c95e None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:59.435740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-1f5d25ee-3a82-439e-af5e-e2936860c95e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:59.435740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:59.435740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:59.435740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:59.435740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:59.435740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:59.435740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:59.435740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:59.435740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:59.435740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:59.435740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:59.435740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:59.435740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:13:59.435740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:13:59.435740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:59.436990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:59.438345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:13:59.438345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:13:59.438345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:13:59.438345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:13:59.438345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 635/1278] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:59.443084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957353c-27b4-4242-92b5-30cc7df611cd None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:59.558084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957353c-27b4-4242-92b5-30cc7df611cd None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:59.598299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ccd76-63a1-459e-82cf-efb3c5cf7fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:59.598923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ccd76-63a1-459e-82cf-efb3c5cf7fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:59.599196 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ccd76-63a1-459e-82cf-efb3c5cf7fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:59.599466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ccd76-63a1-459e-82cf-efb3c5cf7fd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:59.600164 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ccd76-63a1-459e-82cf-efb3c5cf7fd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:59.600464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ccd76-63a1-459e-82cf-efb3c5cf7fd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:59.601940 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ccd76-63a1-459e-82cf-efb3c5cf7fd3 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:59.614767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 644/1279] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:13:59.623447 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d25ee-3a82-439e-af5e-e2936860c95e None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:59.654898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:59.736983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d25ee-3a82-439e-af5e-e2936860c95e None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:59.776102 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79235e46-1f0a-4f66-baa1-953253521a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:59.776771 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79235e46-1f0a-4f66-baa1-953253521a38 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:59.777054 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79235e46-1f0a-4f66-baa1-953253521a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:59.777351 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79235e46-1f0a-4f66-baa1-953253521a38 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:59.790625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79235e46-1f0a-4f66-baa1-953253521a38 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:59.792858 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79235e46-1f0a-4f66-baa1-953253521a38 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:13:59.807027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-79235e46-1f0a-4f66-baa1-953253521a38 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:13:59.807027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:13:59.807027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:13:59.807027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:13:59.807027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:13:59.807027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:13:59.807027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:13:59.807027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:13:59.807027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:13:59.807027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:13:59.807027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:13:59.807027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:13:59.807027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:13:59.807027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:13:59.807027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:13:59.807027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:13:59.808188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:13:59.809376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:13:59.809376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:13:59.809376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:13:59.809376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:13:59.809376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:13:59.809376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:13:59.809376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:13:59.809376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:13:59.809376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:13:59.809376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:13:59.809376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 636/1280] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:13:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:13:59.815096 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ccd76-63a1-459e-82cf-efb3c5cf7fd3 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:13:59.821546 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:13:59.943560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ccd76-63a1-459e-82cf-efb3c5cf7fd3 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:13:59.979832 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6a365-a239-4918-8bf2-ca82cb30333f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:13:59.980450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6a365-a239-4918-8bf2-ca82cb30333f None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:13:59.980720 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6a365-a239-4918-8bf2-ca82cb30333f None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:13:59.980986 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6a365-a239-4918-8bf2-ca82cb30333f None admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:13:59.981819 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6a365-a239-4918-8bf2-ca82cb30333f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:13:59.982099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6a365-a239-4918-8bf2-ca82cb30333f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:13:59.983527 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6a365-a239-4918-8bf2-ca82cb30333f None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:00.001973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 645/1281] 10.222.0.48 () {56 vars in 1143 bytes} [Mon Aug 1 15:13:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:00.007846 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79235e46-1f0a-4f66-baa1-953253521a38 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:00.115792 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:00.125887 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79235e46-1f0a-4f66-baa1-953253521a38 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:00.163810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269e091-20d2-43a6-96fc-37510b2be566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:00.164352 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269e091-20d2-43a6-96fc-37510b2be566 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:00.164624 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269e091-20d2-43a6-96fc-37510b2be566 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:00.164893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269e091-20d2-43a6-96fc-37510b2be566 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:00.243523 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269e091-20d2-43a6-96fc-37510b2be566 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:00.244978 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269e091-20d2-43a6-96fc-37510b2be566 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:00.349838 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:00.361730 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 637/1282] 10.222.0.48 () {58 vars in 1282 bytes} [Mon Aug 1 15:14:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:14:00.367780 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6a365-a239-4918-8bf2-ca82cb30333f None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:00.435214 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:00.479553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6a365-a239-4918-8bf2-ca82cb30333f None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:00.522823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561fe3a-c420-447c-8859-23cfbaba02a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:00.523473 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561fe3a-c420-447c-8859-23cfbaba02a3 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:00.523749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561fe3a-c420-447c-8859-23cfbaba02a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:00.524022 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561fe3a-c420-447c-8859-23cfbaba02a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:00.535283 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561fe3a-c420-447c-8859-23cfbaba02a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:00.536745 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561fe3a-c420-447c-8859-23cfbaba02a3 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:00.552360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-8561fe3a-c420-447c-8859-23cfbaba02a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:14:00.552360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:14:00.552360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:14:00.552360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:14:00.552360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:14:00.552360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:14:00.552360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:14:00.552360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:14:00.552360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:14:00.552360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:14:00.552360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:14:00.552360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:14:00.552360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:14:00.552360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:14:00.552360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:14:00.553328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:14:00.554457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:14:00.554457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:14:00.554457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:14:00.554457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:14:00.554457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 646/1283] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:14:00.560039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269e091-20d2-43a6-96fc-37510b2be566 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:00.670165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269e091-20d2-43a6-96fc-37510b2be566 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:00.713726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2c18f-dea4-4c49-bec3-9591178ee6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:00.714288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2c18f-dea4-4c49-bec3-9591178ee6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:00.714571 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2c18f-dea4-4c49-bec3-9591178ee6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:00.714844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2c18f-dea4-4c49-bec3-9591178ee6c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:00.715538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2c18f-dea4-4c49-bec3-9591178ee6c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:00.715839 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2c18f-dea4-4c49-bec3-9591178ee6c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:00.717218 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2c18f-dea4-4c49-bec3-9591178ee6c0 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:00.730009 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 638/1284] 10.222.0.48 () {56 vars in 1109 bytes} [Mon Aug 1 15:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:00.736096 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561fe3a-c420-447c-8859-23cfbaba02a3 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:00.842961 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561fe3a-c420-447c-8859-23cfbaba02a3 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:00.858144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:00.884848 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86f257-406f-4416-9bef-ff4190f57cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:00.885502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86f257-406f-4416-9bef-ff4190f57cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:00.885804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86f257-406f-4416-9bef-ff4190f57cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:00.886087 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86f257-406f-4416-9bef-ff4190f57cc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:00.892737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86f257-406f-4416-9bef-ff4190f57cc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:00.894862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86f257-406f-4416-9bef-ff4190f57cc4 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:00.906321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application [None req-ba86f257-406f-4416-9bef-ff4190f57cc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:14:00.906321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:14:00.906321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:14:00.906321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:14:00.906321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:14:00.906321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:14:00.906321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:14:00.906321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:14:00.906321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:14:00.906321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:14:00.906321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:14:00.906321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:14:00.906321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:14:00.906321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:14:00.906321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application with Lock( Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:14:00.907339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:14:00.908517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:14:00.908517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:14:00.908517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:14:00.908517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:14:00.908517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:14:00.908517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:14:00.908517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:14:00.908517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: ERROR keystone.server.flask.application Aug 01 15:14:00.908517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 647/1285] 10.222.0.48 () {56 vars in 1111 bytes} [Mon Aug 1 15:14:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:14:00.915794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2c18f-dea4-4c49-bec3-9591178ee6c0 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:00.980376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:01.035260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2c18f-dea4-4c49-bec3-9591178ee6c0 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:01.077706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc1ae9-98c7-4a7a-a24f-ef3d0bb3dd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:01.078656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc1ae9-98c7-4a7a-a24f-ef3d0bb3dd45 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:01.079059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc1ae9-98c7-4a7a-a24f-ef3d0bb3dd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:01.079454 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc1ae9-98c7-4a7a-a24f-ef3d0bb3dd45 None admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:01.080647 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc1ae9-98c7-4a7a-a24f-ef3d0bb3dd45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:01.081102 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc1ae9-98c7-4a7a-a24f-ef3d0bb3dd45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:01.083479 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc1ae9-98c7-4a7a-a24f-ef3d0bb3dd45 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:01.100890 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 639/1286] 10.222.0.48 () {56 vars in 1148 bytes} [Mon Aug 1 15:14:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:01.107969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86f257-406f-4416-9bef-ff4190f57cc4 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:01.215028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86f257-406f-4416-9bef-ff4190f57cc4 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:01.249974 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c7884-b91e-44b5-882b-3a3a2309a0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:01.250512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c7884-b91e-44b5-882b-3a3a2309a0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:01.250790 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c7884-b91e-44b5-882b-3a3a2309a0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:01.251062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c7884-b91e-44b5-882b-3a3a2309a0e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:01.251888 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c7884-b91e-44b5-882b-3a3a2309a0e8 None admin] RBAC: Adding query filter params (role.id=4db5fcc48be54ef597f2e2fffeae9018, scope.project.id=218a9ee5cb2842248b6995d2562bb74b, user.id=098d8d7e35804ccc931ddfba07f688a5) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:01.252234 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c7884-b91e-44b5-882b-3a3a2309a0e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:01.253713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c7884-b91e-44b5-882b-3a3a2309a0e8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:01.259731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 648/1287] 10.222.0.48 () {56 vars in 1374 bytes} [Mon Aug 1 15:14:01 2022] GET /identity/v3/role_assignments?user.id=098d8d7e35804ccc931ddfba07f688a5&scope.project.id=218a9ee5cb2842248b6995d2562bb74b&role.id=4db5fcc48be54ef597f2e2fffeae9018 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:03.247048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7ce63-ebb6-408b-8141-f9af58683b46 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:03.247573 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7ce63-ebb6-408b-8141-f9af58683b46 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:03.247573 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7ce63-ebb6-408b-8141-f9af58683b46 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:03.248855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 640/1288] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:14:03.254333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa6cad-9915-4b40-ba25-667e284187b4 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:03.254766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa6cad-9915-4b40-ba25-667e284187b4 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:03.254866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa6cad-9915-4b40-ba25-667e284187b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:03.274964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-67fa6cad-9915-4b40-ba25-667e284187b4 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:03.276160 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-67fa6cad-9915-4b40-ba25-667e284187b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:03.317122 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:03.365935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 649/1289] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:14:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:14:03.394492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7ce63-ebb6-408b-8141-f9af58683b46 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:03.502943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7ce63-ebb6-408b-8141-f9af58683b46 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:03.514694 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:03.542957 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf487aa-bcb4-4a51-b626-ab7334eb95ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:03.543871 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf487aa-bcb4-4a51-b626-ab7334eb95ad None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:03.544254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf487aa-bcb4-4a51-b626-ab7334eb95ad None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:03.544622 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf487aa-bcb4-4a51-b626-ab7334eb95ad None admin] PATH_INFO: `/v3/services/placement` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:03.545626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf487aa-bcb4-4a51-b626-ab7334eb95ad None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:03.547718 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf487aa-bcb4-4a51-b626-ab7334eb95ad None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-3bf487aa-bcb4-4a51-b626-ab7334eb95ad None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:14:03.557491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:14:03.558789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:14:03.558789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:14:03.558789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:14:03.558789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:14:03.558789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:14:03.558789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:14:03.558789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:14:03.558789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:14:03.558789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:14:03.558789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:14:03.558789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:14:03.558789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:14:03.558789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:14:03.558789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:14:03.558789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:14:03.558789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:14:03.559683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:14:03.559683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:14:03.559683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:14:03.559683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:14:03.559683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:14:03.559683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:14:03.559683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:14:03.559683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:14:03.559683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:14:03.559683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:14:03.559683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:14:03.559683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:14:03.559683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:14:03.559683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:14:03.559683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 641/1290] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 15:14:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:14:03.564984 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa6cad-9915-4b40-ba25-667e284187b4 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:03.670413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa6cad-9915-4b40-ba25-667e284187b4 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:03.710123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14406667-13f9-49dd-98a8-49ffc1a3a478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:03.710742 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14406667-13f9-49dd-98a8-49ffc1a3a478 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:03.711022 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14406667-13f9-49dd-98a8-49ffc1a3a478 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:03.711290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14406667-13f9-49dd-98a8-49ffc1a3a478 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:03.711970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14406667-13f9-49dd-98a8-49ffc1a3a478 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:03.712277 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14406667-13f9-49dd-98a8-49ffc1a3a478 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:03.713663 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14406667-13f9-49dd-98a8-49ffc1a3a478 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:03.718706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 650/1291] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Aug 1 15:14:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:03.724893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf487aa-bcb4-4a51-b626-ab7334eb95ad None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:03.849275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf487aa-bcb4-4a51-b626-ab7334eb95ad None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:03.887690 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a2bb9-3dfa-4da8-b5a5-0b9862b9f593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:03.888315 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a2bb9-3dfa-4da8-b5a5-0b9862b9f593 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:03.888625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a2bb9-3dfa-4da8-b5a5-0b9862b9f593 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:03.888890 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a2bb9-3dfa-4da8-b5a5-0b9862b9f593 None admin] PATH_INFO: `/v3/services` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:03.889568 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a2bb9-3dfa-4da8-b5a5-0b9862b9f593 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:03.889890 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a2bb9-3dfa-4da8-b5a5-0b9862b9f593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:03.891269 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a2bb9-3dfa-4da8-b5a5-0b9862b9f593 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:03.896234 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 642/1292] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Aug 1 15:14:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:05.843463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cf78b-7676-4347-8cdc-c7ccdb187bb0 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:05.844018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cf78b-7676-4347-8cdc-c7ccdb187bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:05.844018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cf78b-7676-4347-8cdc-c7ccdb187bb0 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:05.845383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 651/1293] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:14:05.851785 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1a31b-79b1-49bf-b1a3-eccdb91c2e29 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:05.852505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1a31b-79b1-49bf-b1a3-eccdb91c2e29 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:05.852690 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1a31b-79b1-49bf-b1a3-eccdb91c2e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:05.875184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-71b1a31b-79b1-49bf-b1a3-eccdb91c2e29 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:05.876398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-71b1a31b-79b1-49bf-b1a3-eccdb91c2e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:05.917441 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:05.966217 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 643/1294] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:14:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:14:05.989690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cf78b-7676-4347-8cdc-c7ccdb187bb0 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:06.027943 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:06.111488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cf78b-7676-4347-8cdc-c7ccdb187bb0 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:06.150581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779dd294-1619-42d6-acb3-0504287c18d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:06.151192 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779dd294-1619-42d6-acb3-0504287c18d8 None admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:06.151470 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779dd294-1619-42d6-acb3-0504287c18d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:06.151738 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779dd294-1619-42d6-acb3-0504287c18d8 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:06.152421 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779dd294-1619-42d6-acb3-0504287c18d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:06.153828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779dd294-1619-42d6-acb3-0504287c18d8 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:06.171775 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 652/1295] 10.222.0.48 () {60 vars in 1141 bytes} [Mon Aug 1 15:14:05 2022] POST /identity/v3/services => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:08.169122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83553964-b618-43e6-af2d-3962344fb5b6 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:08.169666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83553964-b618-43e6-af2d-3962344fb5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:08.169666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83553964-b618-43e6-af2d-3962344fb5b6 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:08.170970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 644/1296] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:14:08.177324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114143a4-1cf1-4c0e-a08d-9a3f4a538482 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:08.178106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114143a4-1cf1-4c0e-a08d-9a3f4a538482 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:08.178263 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114143a4-1cf1-4c0e-a08d-9a3f4a538482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:08.207388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-114143a4-1cf1-4c0e-a08d-9a3f4a538482 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:08.208599 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-114143a4-1cf1-4c0e-a08d-9a3f4a538482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:08.285801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 653/1297] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:14:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:14:08.312430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83553964-b618-43e6-af2d-3962344fb5b6 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:08.424268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83553964-b618-43e6-af2d-3962344fb5b6 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:08.461486 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20a7ae-7d78-4dac-8aa4-d5a6c0828aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:08.462159 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20a7ae-7d78-4dac-8aa4-d5a6c0828aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:08.462436 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20a7ae-7d78-4dac-8aa4-d5a6c0828aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:08.462703 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20a7ae-7d78-4dac-8aa4-d5a6c0828aa6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:08.463329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20a7ae-7d78-4dac-8aa4-d5a6c0828aa6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:08.464706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee20a7ae-7d78-4dac-8aa4-d5a6c0828aa6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-ee20a7ae-7d78-4dac-8aa4-d5a6c0828aa6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:14:08.474191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:14:08.475366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:14:08.475366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:14:08.475366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:14:08.475366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:14:08.475366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:14:08.475366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:14:08.475366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:14:08.475366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:14:08.475366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:14:08.475366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:14:08.475366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:14:08.475366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:14:08.475366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:14:08.475366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:14:08.475366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:14:08.475366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:14:08.476240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:14:08.476240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:14:08.476240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:14:08.476240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:14:08.476240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:14:08.476240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:14:08.476240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:14:08.476240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:14:08.476240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:14:08.476240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:14:08.476240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:14:08.476240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:14:08.476240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:14:08.476240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:14:08.476240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 645/1298] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 15:14:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:14:08.484507 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114143a4-1cf1-4c0e-a08d-9a3f4a538482 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:08.509904 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:08.594377 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114143a4-1cf1-4c0e-a08d-9a3f4a538482 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:08.630250 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7774043-c056-4969-a2ba-65e1e2bd25c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:08.630870 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7774043-c056-4969-a2ba-65e1e2bd25c9 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:08.631148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7774043-c056-4969-a2ba-65e1e2bd25c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:08.631418 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7774043-c056-4969-a2ba-65e1e2bd25c9 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:08.632097 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7774043-c056-4969-a2ba-65e1e2bd25c9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:08.632417 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7774043-c056-4969-a2ba-65e1e2bd25c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:08.633803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7774043-c056-4969-a2ba-65e1e2bd25c9 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:08.639072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 654/1299] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Aug 1 15:14:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:08.645455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20a7ae-7d78-4dac-8aa4-d5a6c0828aa6 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:08.681567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:08.756868 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee20a7ae-7d78-4dac-8aa4-d5a6c0828aa6 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:08.800696 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20a67a-b750-4848-bdf6-72099bf5209b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:08.801627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20a67a-b750-4848-bdf6-72099bf5209b None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:08.802078 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20a67a-b750-4848-bdf6-72099bf5209b None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:08.802485 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20a67a-b750-4848-bdf6-72099bf5209b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:08.803720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20a67a-b750-4848-bdf6-72099bf5209b None admin] RBAC: Adding query filter params (interface=public, service_id=da1e63ed19b7465aa3519abda1cb788c, region_id=RegionOne) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:08.804185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20a67a-b750-4848-bdf6-72099bf5209b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:08.806419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20a67a-b750-4848-bdf6-72099bf5209b None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:08.814996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 646/1300] 10.222.0.48 () {56 vars in 1251 bytes} [Mon Aug 1 15:14:08 2022] GET /identity/v3/endpoints?service_id=da1e63ed19b7465aa3519abda1cb788c&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:08.824053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7774043-c056-4969-a2ba-65e1e2bd25c9 None admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:08.931121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:08.951625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7774043-c056-4969-a2ba-65e1e2bd25c9 None admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:08.987142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09a935-6790-4846-842d-866d0dd5ce61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:08.987760 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09a935-6790-4846-842d-866d0dd5ce61 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:08.988037 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09a935-6790-4846-842d-866d0dd5ce61 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:08.988308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09a935-6790-4846-842d-866d0dd5ce61 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:08.988968 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af09a935-6790-4846-842d-866d0dd5ce61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:08.990389 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af09a935-6790-4846-842d-866d0dd5ce61 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:08.997434 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 655/1301] 10.222.0.48 () {56 vars in 1087 bytes} [Mon Aug 1 15:14:08 2022] GET /identity/v3/services => generated 2191 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:10.985458 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710ae64-52e9-47f6-8526-4cea083f023a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:10.986355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710ae64-52e9-47f6-8526-4cea083f023a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:10.986355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710ae64-52e9-47f6-8526-4cea083f023a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:10.988555 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 647/1302] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:14:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:14:10.993901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616c042-a2c5-48d7-a503-be8001834f20 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:10.994371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616c042-a2c5-48d7-a503-be8001834f20 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:10.994462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616c042-a2c5-48d7-a503-be8001834f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:11.015495 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-b616c042-a2c5-48d7-a503-be8001834f20 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:11.016771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-b616c042-a2c5-48d7-a503-be8001834f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:11.092445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 656/1303] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:14:10 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:14:11.116087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710ae64-52e9-47f6-8526-4cea083f023a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:11.240632 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:11.248809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710ae64-52e9-47f6-8526-4cea083f023a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:11.289466 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f916e-7e45-49b4-af35-dd636ab9eb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:11.290389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f916e-7e45-49b4-af35-dd636ab9eb51 None admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:11.290873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f916e-7e45-49b4-af35-dd636ab9eb51 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:11.291290 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f916e-7e45-49b4-af35-dd636ab9eb51 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:11.292005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f916e-7e45-49b4-af35-dd636ab9eb51 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:11.293656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f916e-7e45-49b4-af35-dd636ab9eb51 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application [None req-272f916e-7e45-49b4-af35-dd636ab9eb51 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:14:11.306615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:14:11.308136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:14:11.308136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:14:11.308136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:14:11.308136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:14:11.308136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:14:11.308136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:14:11.308136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:14:11.308136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:14:11.308136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:14:11.308136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:14:11.308136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:14:11.308136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:14:11.308136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:14:11.308136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:14:11.308136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:14:11.308136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:14:11.309263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:14:11.309263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application with Lock( Aug 01 15:14:11.309263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:14:11.309263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:14:11.309263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:14:11.309263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:14:11.309263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:14:11.309263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:14:11.309263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:14:11.309263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:14:11.309263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:14:11.309263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:14:11.309263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:14:11.309263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: ERROR keystone.server.flask.application Aug 01 15:14:11.309263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 648/1304] 10.222.0.48 () {56 vars in 1117 bytes} [Mon Aug 1 15:14:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:14:11.317006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616c042-a2c5-48d7-a503-be8001834f20 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:11.402512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:11.427123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616c042-a2c5-48d7-a503-be8001834f20 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:11.463144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055de182-5a2a-4422-8182-3905eb5a2e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:11.463745 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055de182-5a2a-4422-8182-3905eb5a2e42 None admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:11.464015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055de182-5a2a-4422-8182-3905eb5a2e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:11.464284 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055de182-5a2a-4422-8182-3905eb5a2e42 None admin] PATH_INFO: `/v3/services` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:11.464956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055de182-5a2a-4422-8182-3905eb5a2e42 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:11.465249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055de182-5a2a-4422-8182-3905eb5a2e42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:11.466637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055de182-5a2a-4422-8182-3905eb5a2e42 None admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:11.472007 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 657/1305] 10.222.0.48 () {56 vars in 1116 bytes} [Mon Aug 1 15:14:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:11.478716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f916e-7e45-49b4-af35-dd636ab9eb51 None admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:11.599756 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f916e-7e45-49b4-af35-dd636ab9eb51 None admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:11.647360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9accf-7929-4126-a6e9-871e7ae428d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:11.647982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9accf-7929-4126-a6e9-871e7ae428d6 None admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:11.648262 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9accf-7929-4126-a6e9-871e7ae428d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:11.648536 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9accf-7929-4126-a6e9-871e7ae428d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:11.649248 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9accf-7929-4126-a6e9-871e7ae428d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:11.650671 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9accf-7929-4126-a6e9-871e7ae428d6 None admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:11.679326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 649/1306] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:14:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:21.469083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=30 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:21.671280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:16.420210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d8ce5-4465-4770-85da-86055a57beb0 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.420684 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d8ce5-4465-4770-85da-86055a57beb0 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.420771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d8ce5-4465-4770-85da-86055a57beb0 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.426309 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 658/1307] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:15:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:15:16.427790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9aaa1-4ec3-4aba-85d7-7ead22574a17 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.428206 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9aaa1-4ec3-4aba-85d7-7ead22574a17 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.428439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9aaa1-4ec3-4aba-85d7-7ead22574a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.532250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-99d9aaa1-4ec3-4aba-85d7-7ead22574a17 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:16.536646 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-99d9aaa1-4ec3-4aba-85d7-7ead22574a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:16.678938 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 650/1308] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:16.743952 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bdd83-7f20-4839-ade4-9dc764e75f89 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.744266 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bdd83-7f20-4839-ade4-9dc764e75f89 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.744785 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bdd83-7f20-4839-ade4-9dc764e75f89 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.746773 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 659/1309] 10.222.0.48 () {54 vars in 965 bytes} [Mon Aug 1 15:15:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:15:16.756839 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff2de5-2867-490e-9c23-067d048e6da2 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.757487 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff2de5-2867-490e-9c23-067d048e6da2 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.758004 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff2de5-2867-490e-9c23-067d048e6da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.860047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a2ff2de5-2867-490e-9c23-067d048e6da2 None None] MFA Rules not processed for user `5ef6577b10f7426a925d594e627f27cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:16.861555 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a2ff2de5-2867-490e-9c23-067d048e6da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:16.891726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:16.959146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 651/1310] 10.222.0.48 () {58 vars in 1062 bytes} [Mon Aug 1 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:16.968191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bdd83-7f20-4839-ade4-9dc764e75f89 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:16.970168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1b1bdd83-7f20-4839-ade4-9dc764e75f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:17.112161 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bdd83-7f20-4839-ade4-9dc764e75f89 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:17.203419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61104c1b-19a9-4438-adbc-85739213443a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:17.204107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61104c1b-19a9-4438-adbc-85739213443a service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:17.204385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61104c1b-19a9-4438-adbc-85739213443a service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:17.204688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61104c1b-19a9-4438-adbc-85739213443a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:17.206465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-61104c1b-19a9-4438-adbc-85739213443a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:17.270583 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:17.333182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61104c1b-19a9-4438-adbc-85739213443a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:17.334790 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61104c1b-19a9-4438-adbc-85739213443a service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:17.357471 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 660/1311] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:19.762681 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c85871-ff0f-4686-b25e-caf65f83880f None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:19.763259 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c85871-ff0f-4686-b25e-caf65f83880f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:19.763259 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c85871-ff0f-4686-b25e-caf65f83880f None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:19.764756 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 652/1312] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:15:19.770113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bb471-fd52-4372-b136-8a0faeb16ada None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:19.770597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bb471-fd52-4372-b136-8a0faeb16ada None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:19.770756 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bb471-fd52-4372-b136-8a0faeb16ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:19.860415 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-329bb471-fd52-4372-b136-8a0faeb16ada None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:19.861722 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-329bb471-fd52-4372-b136-8a0faeb16ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:19.995640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 661/1313] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:20.034535 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c85871-ff0f-4686-b25e-caf65f83880f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:20.036268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e4c85871-ff0f-4686-b25e-caf65f83880f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:20.149988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c85871-ff0f-4686-b25e-caf65f83880f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:20.226024 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:20.229212 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0bbd2-6bcd-4f7a-b356-b5aaf6eb6be6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:20.229836 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0bbd2-6bcd-4f7a-b356-b5aaf6eb6be6 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.230339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0bbd2-6bcd-4f7a-b356-b5aaf6eb6be6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.230617 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0bbd2-6bcd-4f7a-b356-b5aaf6eb6be6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.232777 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-56c0bbd2-6bcd-4f7a-b356-b5aaf6eb6be6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:20.351040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c0bbd2-6bcd-4f7a-b356-b5aaf6eb6be6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:20.352878 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c0bbd2-6bcd-4f7a-b356-b5aaf6eb6be6 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:20.374099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 653/1314] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:22.719306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472da4e1-7a79-4222-9365-7d10dc3f75a7 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:22.720447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472da4e1-7a79-4222-9365-7d10dc3f75a7 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:22.720447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472da4e1-7a79-4222-9365-7d10dc3f75a7 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:22.721082 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 662/1315] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:15:22.727945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37c6f9-f7ba-4cd0-8ede-80040fe11c4d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:22.728389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37c6f9-f7ba-4cd0-8ede-80040fe11c4d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:22.728535 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37c6f9-f7ba-4cd0-8ede-80040fe11c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:22.799545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-1a37c6f9-f7ba-4cd0-8ede-80040fe11c4d None None] MFA Rules not processed for user `a0dfd1dc38b741ebadff99a9c1fa4e99`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:22.800779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1a37c6f9-f7ba-4cd0-8ede-80040fe11c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:22.855799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:22.904569 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 654/1316] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:15:22 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:22.944521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472da4e1-7a79-4222-9365-7d10dc3f75a7 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:22.947145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-472da4e1-7a79-4222-9365-7d10dc3f75a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:22.958700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:23.062011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472da4e1-7a79-4222-9365-7d10dc3f75a7 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:23.142319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ce071-c209-4a9e-a459-551ebfc06698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:23.142953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ce071-c209-4a9e-a459-551ebfc06698 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.143232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ce071-c209-4a9e-a459-551ebfc06698 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.143502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ce071-c209-4a9e-a459-551ebfc06698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:23.145184 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-ba3ce071-c209-4a9e-a459-551ebfc06698 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:23.211396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:23.261088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ce071-c209-4a9e-a459-551ebfc06698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:23.262568 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ce071-c209-4a9e-a459-551ebfc06698 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:23.282666 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 663/1317] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:22 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:25.940509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb218c6b-9b6e-49ce-b47b-b0b146c647fb None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.941177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb218c6b-9b6e-49ce-b47b-b0b146c647fb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.941177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb218c6b-9b6e-49ce-b47b-b0b146c647fb None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.942942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 655/1318] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:15:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:15:25.949894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fad40-f825-44fa-a969-876187897da2 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.950358 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fad40-f825-44fa-a969-876187897da2 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.950480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fad40-f825-44fa-a969-876187897da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.023317 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-c05fad40-f825-44fa-a969-876187897da2 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:26.024617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c05fad40-f825-44fa-a969-876187897da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.134723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 664/1319] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:15:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:26.190287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8449b-a6ea-4266-af5d-7ff9e75d4cee None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.191369 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8449b-a6ea-4266-af5d-7ff9e75d4cee None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.191867 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8449b-a6ea-4266-af5d-7ff9e75d4cee None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.193592 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 656/1320] 10.222.0.48 () {54 vars in 965 bytes} [Mon Aug 1 15:15:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:15:26.199535 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddd395-0b99-402d-b951-0a5ff949b9b0 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.199818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddd395-0b99-402d-b951-0a5ff949b9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.200095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddd395-0b99-402d-b951-0a5ff949b9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.223933 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:26.282949 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-6bddd395-0b99-402d-b951-0a5ff949b9b0 None None] MFA Rules not processed for user `5ef6577b10f7426a925d594e627f27cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:26.284270 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6bddd395-0b99-402d-b951-0a5ff949b9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.372108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 665/1321] 10.222.0.48 () {58 vars in 1062 bytes} [Mon Aug 1 15:15:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:26.381377 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8449b-a6ea-4266-af5d-7ff9e75d4cee None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.383062 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-03a8449b-a6ea-4266-af5d-7ff9e75d4cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.501109 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8449b-a6ea-4266-af5d-7ff9e75d4cee None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.583044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abe163-b55b-4dce-bc81-8cb3ee4aae46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.583671 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abe163-b55b-4dce-bc81-8cb3ee4aae46 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.583953 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abe163-b55b-4dce-bc81-8cb3ee4aae46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.584229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abe163-b55b-4dce-bc81-8cb3ee4aae46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.586022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c9abe163-b55b-4dce-bc81-8cb3ee4aae46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.593682 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:26.707189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abe163-b55b-4dce-bc81-8cb3ee4aae46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.708659 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abe163-b55b-4dce-bc81-8cb3ee4aae46 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.728020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 657/1322] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:29.938240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0fdc8-f320-4cfe-b5ef-bb07b0c1d16c None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:29.938811 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0fdc8-f320-4cfe-b5ef-bb07b0c1d16c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:29.938811 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0fdc8-f320-4cfe-b5ef-bb07b0c1d16c None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:29.940126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 666/1323] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:15:29.946036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24879c-f6b5-48a1-a25b-440ca3433475 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:29.946825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24879c-f6b5-48a1-a25b-440ca3433475 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:29.946825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24879c-f6b5-48a1-a25b-440ca3433475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:30.018044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-fb24879c-f6b5-48a1-a25b-440ca3433475 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:30.019311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-fb24879c-f6b5-48a1-a25b-440ca3433475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:30.088549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:30.133720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 658/1324] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:30.178976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0fdc8-f320-4cfe-b5ef-bb07b0c1d16c None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:30.180573 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-59b0fdc8-f320-4cfe-b5ef-bb07b0c1d16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:30.322669 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0fdc8-f320-4cfe-b5ef-bb07b0c1d16c None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:30.335614 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:30.403989 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84901f42-89a7-408e-bd76-ee8dfb1787a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:30.404628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84901f42-89a7-408e-bd76-ee8dfb1787a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:30.404902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84901f42-89a7-408e-bd76-ee8dfb1787a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:30.405174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84901f42-89a7-408e-bd76-ee8dfb1787a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:30.406889 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-84901f42-89a7-408e-bd76-ee8dfb1787a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:30.539186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84901f42-89a7-408e-bd76-ee8dfb1787a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:30.540869 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84901f42-89a7-408e-bd76-ee8dfb1787a1 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:30.560116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 667/1325] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:32.619242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edee7bd-a7b2-4952-91c1-131cd8b7d177 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:32.619796 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edee7bd-a7b2-4952-91c1-131cd8b7d177 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:32.619796 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edee7bd-a7b2-4952-91c1-131cd8b7d177 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:32.621180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 659/1326] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:15:32.626818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e0fd9-baeb-4941-8d32-b8f51ba97d59 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:32.627235 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e0fd9-baeb-4941-8d32-b8f51ba97d59 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:32.627490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e0fd9-baeb-4941-8d32-b8f51ba97d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:32.682020 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:32.701503 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-0a2e0fd9-baeb-4941-8d32-b8f51ba97d59 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:32.702691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-0a2e0fd9-baeb-4941-8d32-b8f51ba97d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:32.811125 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 668/1327] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:32.847065 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edee7bd-a7b2-4952-91c1-131cd8b7d177 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:32.848722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5edee7bd-a7b2-4952-91c1-131cd8b7d177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:32.962499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edee7bd-a7b2-4952-91c1-131cd8b7d177 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:33.038928 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d7d22-0ebf-4af7-9c4e-402e6b3b899c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:33.039552 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d7d22-0ebf-4af7-9c4e-402e6b3b899c service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:33.039832 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d7d22-0ebf-4af7-9c4e-402e6b3b899c service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:33.040105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d7d22-0ebf-4af7-9c4e-402e6b3b899c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:33.041761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5d6d7d22-0ebf-4af7-9c4e-402e6b3b899c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:33.060071 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:33.159345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d7d22-0ebf-4af7-9c4e-402e6b3b899c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:33.160813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d7d22-0ebf-4af7-9c4e-402e6b3b899c service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:33.180114 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 660/1328] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:35.227850 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb5785-ae3d-4cfe-b999-126168b0d0f5 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:35.228450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb5785-ae3d-4cfe-b999-126168b0d0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:35.228450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb5785-ae3d-4cfe-b999-126168b0d0f5 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:35.230164 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 669/1329] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:15:35.236865 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dd378-693c-4b5f-b18f-d7a73b5f0d81 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:35.237525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dd378-693c-4b5f-b18f-d7a73b5f0d81 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:35.237525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dd378-693c-4b5f-b18f-d7a73b5f0d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:35.311381 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b02dd378-693c-4b5f-b18f-d7a73b5f0d81 None None] MFA Rules not processed for user `a0dfd1dc38b741ebadff99a9c1fa4e99`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:35.312589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b02dd378-693c-4b5f-b18f-d7a73b5f0d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:35.385312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:35.420627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 661/1330] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:15:35 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:35.459322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb5785-ae3d-4cfe-b999-126168b0d0f5 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:35.460896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-0deb5785-ae3d-4cfe-b999-126168b0d0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:35.575087 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb5785-ae3d-4cfe-b999-126168b0d0f5 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:35.596433 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:35.655671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbfefc4-48cf-4b4f-adaf-79887f8222cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:35.656329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfefc4-48cf-4b4f-adaf-79887f8222cf service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:35.656602 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfefc4-48cf-4b4f-adaf-79887f8222cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:35.656890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfefc4-48cf-4b4f-adaf-79887f8222cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:35.658546 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-7dbfefc4-48cf-4b4f-adaf-79887f8222cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:35.779965 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbfefc4-48cf-4b4f-adaf-79887f8222cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:35.781433 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbfefc4-48cf-4b4f-adaf-79887f8222cf service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:35.801074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 670/1331] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:38.091978 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08484958-53c5-43cd-b381-2d042fd0e25f None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:38.092576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08484958-53c5-43cd-b381-2d042fd0e25f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:38.092576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08484958-53c5-43cd-b381-2d042fd0e25f None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:38.094008 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 662/1332] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:15:38.099550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c87c62-f17b-40d9-94e3-4ee6e565e8eb None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:38.099999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c87c62-f17b-40d9-94e3-4ee6e565e8eb None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:38.100108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c87c62-f17b-40d9-94e3-4ee6e565e8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:38.166631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:38.169672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-15c87c62-f17b-40d9-94e3-4ee6e565e8eb None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:38.171737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-15c87c62-f17b-40d9-94e3-4ee6e565e8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:38.281175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 671/1333] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:38.326602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08484958-53c5-43cd-b381-2d042fd0e25f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:38.328187 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-08484958-53c5-43cd-b381-2d042fd0e25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:38.441092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08484958-53c5-43cd-b381-2d042fd0e25f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:38.535488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301af77-f799-4e1e-b564-2784a1b652d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:38.536096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301af77-f799-4e1e-b564-2784a1b652d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:38.536370 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301af77-f799-4e1e-b564-2784a1b652d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:38.536640 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301af77-f799-4e1e-b564-2784a1b652d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:38.538457 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1301af77-f799-4e1e-b564-2784a1b652d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:38.579456 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:38.668287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301af77-f799-4e1e-b564-2784a1b652d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:38.669922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301af77-f799-4e1e-b564-2784a1b652d8 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:38.688879 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 663/1334] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:40.741833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb368b52-7e14-4a43-badf-602394e59244 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:40.742403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb368b52-7e14-4a43-badf-602394e59244 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:40.742403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb368b52-7e14-4a43-badf-602394e59244 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:40.743765 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 672/1335] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:15:40.749362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b01377-6731-4777-b5dc-ee85101787cc None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:40.749898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b01377-6731-4777-b5dc-ee85101787cc None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:40.749994 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b01377-6731-4777-b5dc-ee85101787cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:40.820900 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-38b01377-6731-4777-b5dc-ee85101787cc None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:40.822165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-38b01377-6731-4777-b5dc-ee85101787cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:40.907445 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:40.934635 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 664/1336] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:40.977311 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb368b52-7e14-4a43-badf-602394e59244 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:40.979075 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-fb368b52-7e14-4a43-badf-602394e59244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:41.091528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb368b52-7e14-4a43-badf-602394e59244 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:41.120934 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:41.167628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a406d0-1114-4fd2-946c-4f1210a19b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:41.168251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a406d0-1114-4fd2-946c-4f1210a19b0b service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:41.168530 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a406d0-1114-4fd2-946c-4f1210a19b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:41.168804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a406d0-1114-4fd2-946c-4f1210a19b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:41.170477 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-84a406d0-1114-4fd2-946c-4f1210a19b0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:41.285799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a406d0-1114-4fd2-946c-4f1210a19b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:41.287334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a406d0-1114-4fd2-946c-4f1210a19b0b service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:41.307872 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 673/1337] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:43.776181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafc597-2c79-4516-b33b-5ec25d816615 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:43.776757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafc597-2c79-4516-b33b-5ec25d816615 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:43.776757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafc597-2c79-4516-b33b-5ec25d816615 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:43.778132 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 665/1338] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:15:43.787109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8345930-70db-4e27-8954-ad0f7c17d080 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:43.787858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8345930-70db-4e27-8954-ad0f7c17d080 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:43.788026 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8345930-70db-4e27-8954-ad0f7c17d080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:43.864945 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-d8345930-70db-4e27-8954-ad0f7c17d080 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:43.866221 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d8345930-70db-4e27-8954-ad0f7c17d080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:43.876355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:43.983426 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 674/1339] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:15:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:44.034830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafc597-2c79-4516-b33b-5ec25d816615 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:44.037115 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0dafc597-2c79-4516-b33b-5ec25d816615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:44.165468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafc597-2c79-4516-b33b-5ec25d816615 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:44.246250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0cae6-5f59-4049-8840-62d428acdb5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:44.246796 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0cae6-5f59-4049-8840-62d428acdb5f service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:44.247079 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0cae6-5f59-4049-8840-62d428acdb5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:44.247723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0cae6-5f59-4049-8840-62d428acdb5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:44.249549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c6d0cae6-5f59-4049-8840-62d428acdb5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:44.292437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:44.376669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0cae6-5f59-4049-8840-62d428acdb5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:44.378310 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0cae6-5f59-4049-8840-62d428acdb5f service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:44.398772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 666/1340] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:48.424644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69985e76-714b-4af9-b86e-8b5a7575a17e None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:48.425186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69985e76-714b-4af9-b86e-8b5a7575a17e None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:48.425186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69985e76-714b-4af9-b86e-8b5a7575a17e None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:48.426598 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 675/1341] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:15:48.431871 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59decaba-f154-4917-abae-5248a63409da None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:48.432316 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59decaba-f154-4917-abae-5248a63409da None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:48.432413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59decaba-f154-4917-abae-5248a63409da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:48.501724 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-59decaba-f154-4917-abae-5248a63409da None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:48.502887 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-59decaba-f154-4917-abae-5248a63409da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:48.593770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:48.605737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 667/1342] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:48.645474 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69985e76-714b-4af9-b86e-8b5a7575a17e None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:48.647110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-69985e76-714b-4af9-b86e-8b5a7575a17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:48.761521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69985e76-714b-4af9-b86e-8b5a7575a17e None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:48.802038 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:48.841235 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b21546-0eec-4b5f-9ad6-2c9297faaf64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:48.841906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b21546-0eec-4b5f-9ad6-2c9297faaf64 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:48.842356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b21546-0eec-4b5f-9ad6-2c9297faaf64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:48.842629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b21546-0eec-4b5f-9ad6-2c9297faaf64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:48.844252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-21b21546-0eec-4b5f-9ad6-2c9297faaf64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:48.962639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b21546-0eec-4b5f-9ad6-2c9297faaf64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:48.964315 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b21546-0eec-4b5f-9ad6-2c9297faaf64 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:48.983540 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 676/1343] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:51.862750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7afa6-b3bb-4302-949d-f0568a37d377 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:51.863327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7afa6-b3bb-4302-949d-f0568a37d377 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:51.863462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f7afa6-b3bb-4302-949d-f0568a37d377 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:51.865478 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 668/1344] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:15:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:15:51.871132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be8386-fb9d-431f-9c76-a7dcdba692f5 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:51.871644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be8386-fb9d-431f-9c76-a7dcdba692f5 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:51.871644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be8386-fb9d-431f-9c76-a7dcdba692f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:51.942009 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-27be8386-fb9d-431f-9c76-a7dcdba692f5 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:51.943160 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-27be8386-fb9d-431f-9c76-a7dcdba692f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:51.962059 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:52.046284 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 677/1345] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:15:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:52.084946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7afa6-b3bb-4302-949d-f0568a37d377 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:52.087017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-07f7afa6-b3bb-4302-949d-f0568a37d377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:52.212917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f7afa6-b3bb-4302-949d-f0568a37d377 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:52.300889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c653013-6578-4d5a-826e-3d13421d8ab9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:52.301707 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c653013-6578-4d5a-826e-3d13421d8ab9 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:52.302160 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c653013-6578-4d5a-826e-3d13421d8ab9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:52.302277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c653013-6578-4d5a-826e-3d13421d8ab9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:52.304191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7c653013-6578-4d5a-826e-3d13421d8ab9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:52.360763 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:52.445823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c653013-6578-4d5a-826e-3d13421d8ab9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:52.447772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c653013-6578-4d5a-826e-3d13421d8ab9 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:52.473765 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 669/1346] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:56.575587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e618a-b4d1-48d1-b3dd-86be7da27732 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:56.576279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e618a-b4d1-48d1-b3dd-86be7da27732 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:56.576279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e618a-b4d1-48d1-b3dd-86be7da27732 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:56.577682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 678/1347] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:15:56.584190 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d897e4-72cb-430a-a7da-a515b88e792f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:56.584190 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d897e4-72cb-430a-a7da-a515b88e792f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:56.584845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d897e4-72cb-430a-a7da-a515b88e792f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:56.682801 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c4d897e4-72cb-430a-a7da-a515b88e792f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:56.684038 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c4d897e4-72cb-430a-a7da-a515b88e792f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:56.796337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:56.809859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 670/1348] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:56.838331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e618a-b4d1-48d1-b3dd-86be7da27732 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:56.840053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-ed3e618a-b4d1-48d1-b3dd-86be7da27732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:56.960484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e618a-b4d1-48d1-b3dd-86be7da27732 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:57.012927 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:57.043260 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc4986-90d0-4c49-ad7e-cf0650b1f695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:57.043943 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc4986-90d0-4c49-ad7e-cf0650b1f695 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:57.044245 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc4986-90d0-4c49-ad7e-cf0650b1f695 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:57.044503 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc4986-90d0-4c49-ad7e-cf0650b1f695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:57.046317 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-b7bc4986-90d0-4c49-ad7e-cf0650b1f695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:57.176248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc4986-90d0-4c49-ad7e-cf0650b1f695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:57.178187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc4986-90d0-4c49-ad7e-cf0650b1f695 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:57.197857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 679/1349] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:15:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:02.400487 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd44aae-6ef0-459e-9e85-628a160479e4 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:02.400998 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd44aae-6ef0-459e-9e85-628a160479e4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:02.401385 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd44aae-6ef0-459e-9e85-628a160479e4 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:02.403197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 671/1350] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 15:16:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:02.411481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96014e5-4ea7-4f7b-8c63-7e2bc418341d None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:02.412015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96014e5-4ea7-4f7b-8c63-7e2bc418341d None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:02.412015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96014e5-4ea7-4f7b-8c63-7e2bc418341d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:02.522308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-f96014e5-4ea7-4f7b-8c63-7e2bc418341d None None] MFA Rules not processed for user `098d8d7e35804ccc931ddfba07f688a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:02.523831 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f96014e5-4ea7-4f7b-8c63-7e2bc418341d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:02.547959 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:02.674663 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 680/1351] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 15:16:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:07.283441 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413939b9-a2f4-4f65-ad34-9e28f0998580 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:07.285617 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413939b9-a2f4-4f65-ad34-9e28f0998580 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:07.286203 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413939b9-a2f4-4f65-ad34-9e28f0998580 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:07.288104 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 672/1352] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 15:16:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:07.293686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94a221-8729-4f2e-9af9-1c6e6a27acf7 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:07.294761 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94a221-8729-4f2e-9af9-1c6e6a27acf7 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:07.295287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94a221-8729-4f2e-9af9-1c6e6a27acf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:07.382099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-3a94a221-8729-4f2e-9af9-1c6e6a27acf7 None None] MFA Rules not processed for user `098d8d7e35804ccc931ddfba07f688a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:07.383420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-3a94a221-8729-4f2e-9af9-1c6e6a27acf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:07.469812 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:07.512021 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 681/1353] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 15:16:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:08.962044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ec281-1694-4773-9ed2-12d87c2ce621 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.962750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ec281-1694-4773-9ed2-12d87c2ce621 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.962750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ec281-1694-4773-9ed2-12d87c2ce621 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.964620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 673/1354] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 15:16:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:08.969997 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4de2e-f2b7-41b2-afd8-9ce3e0464974 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.970468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4de2e-f2b7-41b2-afd8-9ce3e0464974 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.970663 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4de2e-f2b7-41b2-afd8-9ce3e0464974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.056356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-01a4de2e-f2b7-41b2-afd8-9ce3e0464974 None None] MFA Rules not processed for user `098d8d7e35804ccc931ddfba07f688a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:09.057699 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-01a4de2e-f2b7-41b2-afd8-9ce3e0464974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:09.178111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 682/1355] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 15:16:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:12.456148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03847ce1-f4d4-43f7-9de0-3179c4f04837 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:12.457032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03847ce1-f4d4-43f7-9de0-3179c4f04837 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:12.457467 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03847ce1-f4d4-43f7-9de0-3179c4f04837 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:12.460125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 674/1356] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:16:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:12.466003 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddb165-bad0-49a5-a774-2c93e78972f7 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:12.466567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddb165-bad0-49a5-a774-2c93e78972f7 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:12.466663 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddb165-bad0-49a5-a774-2c93e78972f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:12.516613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:12.563984 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-d9ddb165-bad0-49a5-a774-2c93e78972f7 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:12.565409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d9ddb165-bad0-49a5-a774-2c93e78972f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:12.681014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 683/1357] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:16:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:12.735821 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b263c52-2fd4-4735-97c1-b28ce0d6a618 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:12.736090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b263c52-2fd4-4735-97c1-b28ce0d6a618 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:12.736604 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b263c52-2fd4-4735-97c1-b28ce0d6a618 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:12.738470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 675/1358] 10.222.0.48 () {54 vars in 962 bytes} [Mon Aug 1 15:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:12.746546 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35219e06-9069-4ed4-982c-b3437e434229 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:12.748278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35219e06-9069-4ed4-982c-b3437e434229 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:12.748760 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35219e06-9069-4ed4-982c-b3437e434229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:12.845661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-35219e06-9069-4ed4-982c-b3437e434229 None None] MFA Rules not processed for user `2de0ff99a2b74979a28a8b10e9ebdfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:12.846981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-35219e06-9069-4ed4-982c-b3437e434229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:12.903879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:12.985545 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 684/1359] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Aug 1 15:16:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:12.993232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b263c52-2fd4-4735-97c1-b28ce0d6a618 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:12.994996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7b263c52-2fd4-4735-97c1-b28ce0d6a618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:13.184642 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b263c52-2fd4-4735-97c1-b28ce0d6a618 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:13.272174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:13.340221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b89ae6-d328-47b7-ae87-a8dff3e9b594 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:13.340980 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b89ae6-d328-47b7-ae87-a8dff3e9b594 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:13.341186 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b89ae6-d328-47b7-ae87-a8dff3e9b594 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:13.341451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b89ae6-d328-47b7-ae87-a8dff3e9b594 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:13.343131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d3b89ae6-d328-47b7-ae87-a8dff3e9b594 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:13.444362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b89ae6-d328-47b7-ae87-a8dff3e9b594 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:13.446074 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b89ae6-d328-47b7-ae87-a8dff3e9b594 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:13.464723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 676/1360] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:15.717244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8d9f5-7f19-45f1-85b1-d8912436e5f1 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.717866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8d9f5-7f19-45f1-85b1-d8912436e5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.717866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8d9f5-7f19-45f1-85b1-d8912436e5f1 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.719437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 685/1361] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:15.725340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1086feb-8eb1-4799-ba13-214f86bcb695 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.725822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1086feb-8eb1-4799-ba13-214f86bcb695 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.725939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1086feb-8eb1-4799-ba13-214f86bcb695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.803283 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d1086feb-8eb1-4799-ba13-214f86bcb695 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:15.804511 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d1086feb-8eb1-4799-ba13-214f86bcb695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:15.869920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:15.985973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 677/1362] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:16.026099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8d9f5-7f19-45f1-85b1-d8912436e5f1 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:16.027685 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5ee8d9f5-7f19-45f1-85b1-d8912436e5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:16.220481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8d9f5-7f19-45f1-85b1-d8912436e5f1 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:16.231210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:16.350863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274dbea-4bcb-456e-a293-2a92fb4ff9da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:16.351666 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274dbea-4bcb-456e-a293-2a92fb4ff9da service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:16.351945 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274dbea-4bcb-456e-a293-2a92fb4ff9da service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:16.352210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274dbea-4bcb-456e-a293-2a92fb4ff9da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:16.353975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d274dbea-4bcb-456e-a293-2a92fb4ff9da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:16.454082 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274dbea-4bcb-456e-a293-2a92fb4ff9da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:16.455618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274dbea-4bcb-456e-a293-2a92fb4ff9da service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:16.478345 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 686/1363] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:18.875904 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067a4b6-0ec7-4b89-b94b-dd931445266f None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:18.876966 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067a4b6-0ec7-4b89-b94b-dd931445266f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:18.877460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067a4b6-0ec7-4b89-b94b-dd931445266f None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:18.882186 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 678/1364] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:16:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:18.888339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbeb953-6e4b-4845-8946-e2a9c0d30419 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:18.888827 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbeb953-6e4b-4845-8946-e2a9c0d30419 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:18.889096 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbeb953-6e4b-4845-8946-e2a9c0d30419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:18.906088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:18.962691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-ffbeb953-6e4b-4845-8946-e2a9c0d30419 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:18.963915 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-ffbeb953-6e4b-4845-8946-e2a9c0d30419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:19.087521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 687/1365] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:19.133217 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a5e3c-a60c-47bd-b942-87587b439d85 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:19.133489 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a5e3c-a60c-47bd-b942-87587b439d85 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:19.134071 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a5e3c-a60c-47bd-b942-87587b439d85 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:19.135845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 679/1366] 10.222.0.48 () {54 vars in 962 bytes} [Mon Aug 1 15:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:19.142035 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fead67-988e-40de-8745-430dd94cf8a7 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:19.142308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fead67-988e-40de-8745-430dd94cf8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:19.142584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fead67-988e-40de-8745-430dd94cf8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:19.226946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-39fead67-988e-40de-8745-430dd94cf8a7 None None] MFA Rules not processed for user `2de0ff99a2b74979a28a8b10e9ebdfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:19.228186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-39fead67-988e-40de-8745-430dd94cf8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:19.265601 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:19.371852 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 688/1367] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Aug 1 15:16:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:19.386626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a5e3c-a60c-47bd-b942-87587b439d85 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:19.388266 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2b0a5e3c-a60c-47bd-b942-87587b439d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:19.577955 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a5e3c-a60c-47bd-b942-87587b439d85 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:19.611721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:19.730945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf34dc-2f98-45aa-b9af-bed15296f8e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:19.731633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf34dc-2f98-45aa-b9af-bed15296f8e7 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:19.731916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf34dc-2f98-45aa-b9af-bed15296f8e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:19.732190 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf34dc-2f98-45aa-b9af-bed15296f8e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:19.733888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a9bf34dc-2f98-45aa-b9af-bed15296f8e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:19.853423 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf34dc-2f98-45aa-b9af-bed15296f8e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:19.855143 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf34dc-2f98-45aa-b9af-bed15296f8e7 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:19.883574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 680/1368] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:20.738960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302b793-443f-4c67-8d6b-b7a33c242955 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:20.739534 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302b793-443f-4c67-8d6b-b7a33c242955 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:20.739635 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b302b793-443f-4c67-8d6b-b7a33c242955 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:20.740985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 689/1369] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:20.746658 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db79e8a-888d-4912-923e-a7b3add54c8d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:20.746971 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db79e8a-888d-4912-923e-a7b3add54c8d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:20.747297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db79e8a-888d-4912-923e-a7b3add54c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:20.786237 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:20.829618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5db79e8a-888d-4912-923e-a7b3add54c8d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:20.830933 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5db79e8a-888d-4912-923e-a7b3add54c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:20.949522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 681/1370] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:21.313669 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48c15b-7756-4cbf-a57d-e64bf1e1842f None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:21.314397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48c15b-7756-4cbf-a57d-e64bf1e1842f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:21.314850 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48c15b-7756-4cbf-a57d-e64bf1e1842f None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:21.316772 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 690/1371] 10.222.0.48 () {54 vars in 964 bytes} [Mon Aug 1 15:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:21.322267 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f1bc7-84be-40ce-909a-dcb5941b3687 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:21.322558 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f1bc7-84be-40ce-909a-dcb5941b3687 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:21.322834 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f1bc7-84be-40ce-909a-dcb5941b3687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:21.404036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-428f1bc7-84be-40ce-909a-dcb5941b3687 None None] MFA Rules not processed for user `63824ba9369c4899a4b946588cbdae24`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:21.405798 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-428f1bc7-84be-40ce-909a-dcb5941b3687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:21.461076 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:21.500823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 682/1372] 10.222.0.48 () {58 vars in 1061 bytes} [Mon Aug 1 15:16:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:21.508619 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48c15b-7756-4cbf-a57d-e64bf1e1842f None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:21.510465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8b48c15b-7756-4cbf-a57d-e64bf1e1842f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:21.637691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48c15b-7756-4cbf-a57d-e64bf1e1842f None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:21.693980 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:21.746131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b98218-67f7-4faa-8b3f-d2b282ae63ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:21.746956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b98218-67f7-4faa-8b3f-d2b282ae63ec service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:21.747356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b98218-67f7-4faa-8b3f-d2b282ae63ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:21.747730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b98218-67f7-4faa-8b3f-d2b282ae63ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:21.749589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e2b98218-67f7-4faa-8b3f-d2b282ae63ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:21.883620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b98218-67f7-4faa-8b3f-d2b282ae63ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:21.883620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b98218-67f7-4faa-8b3f-d2b282ae63ec service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:21.901728 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 691/1373] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:22.030032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097344e-4c81-4449-a6d3-2475446bf0f2 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:22.030472 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097344e-4c81-4449-a6d3-2475446bf0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:22.030706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097344e-4c81-4449-a6d3-2475446bf0f2 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:22.032232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 683/1374] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:22.037688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdc3f0-0944-4304-a9cf-d7b1648256ae None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:22.038031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdc3f0-0944-4304-a9cf-d7b1648256ae None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:22.038425 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdc3f0-0944-4304-a9cf-d7b1648256ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:22.137605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-34bdc3f0-0944-4304-a9cf-d7b1648256ae None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:22.140708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-34bdc3f0-0944-4304-a9cf-d7b1648256ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:22.172167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:22.261596 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 692/1375] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:22.305713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097344e-4c81-4449-a6d3-2475446bf0f2 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:22.307509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c097344e-4c81-4449-a6d3-2475446bf0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:22.457597 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c097344e-4c81-4449-a6d3-2475446bf0f2 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:22.512156 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:22.579945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e52c13-5903-4fbf-a7a7-cf9430d67de9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:22.581010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e52c13-5903-4fbf-a7a7-cf9430d67de9 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:22.581288 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e52c13-5903-4fbf-a7a7-cf9430d67de9 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:22.581558 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e52c13-5903-4fbf-a7a7-cf9430d67de9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:22.584420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e3e52c13-5903-4fbf-a7a7-cf9430d67de9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:22.685013 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e52c13-5903-4fbf-a7a7-cf9430d67de9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:22.686672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e52c13-5903-4fbf-a7a7-cf9430d67de9 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:22.705096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 684/1376] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:24.746379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e559e-6191-4170-9c43-8633692fd061 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:24.747045 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e559e-6191-4170-9c43-8633692fd061 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:24.747045 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e559e-6191-4170-9c43-8633692fd061 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:24.749293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 693/1377] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:16:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:24.754950 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c020d6-32f5-4ddf-90e0-e5fdd12d3288 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:24.755429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c020d6-32f5-4ddf-90e0-e5fdd12d3288 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:24.755530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c020d6-32f5-4ddf-90e0-e5fdd12d3288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:24.832323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-69c020d6-32f5-4ddf-90e0-e5fdd12d3288 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:24.833546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-69c020d6-32f5-4ddf-90e0-e5fdd12d3288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:24.841896 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:24.972423 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 685/1378] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:25.017548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e559e-6191-4170-9c43-8633692fd061 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:25.019184 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c45e559e-6191-4170-9c43-8633692fd061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:25.182066 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e559e-6191-4170-9c43-8633692fd061 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:25.185872 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:25.309649 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef81f5-6a07-4fd9-98d5-571763dc8b72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:25.310580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef81f5-6a07-4fd9-98d5-571763dc8b72 service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:25.310904 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef81f5-6a07-4fd9-98d5-571763dc8b72 service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:25.311249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef81f5-6a07-4fd9-98d5-571763dc8b72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:25.313383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d4ef81f5-6a07-4fd9-98d5-571763dc8b72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:25.431956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef81f5-6a07-4fd9-98d5-571763dc8b72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:25.434058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef81f5-6a07-4fd9-98d5-571763dc8b72 service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:25.458805 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 694/1379] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:27.520985 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a06baa-2d37-41e5-ada7-e44389198214 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:27.521633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a06baa-2d37-41e5-ada7-e44389198214 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:27.521633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a06baa-2d37-41e5-ada7-e44389198214 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:27.523026 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 686/1380] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:27.528753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a3d7e-5f1f-4e9a-9d8b-32f0be06be6f None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:27.529277 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a3d7e-5f1f-4e9a-9d8b-32f0be06be6f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:27.529502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a3d7e-5f1f-4e9a-9d8b-32f0be06be6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:27.541986 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:27.605881 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-e82a3d7e-5f1f-4e9a-9d8b-32f0be06be6f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:27.607141 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e82a3d7e-5f1f-4e9a-9d8b-32f0be06be6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:27.721945 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 695/1381] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:27.763186 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a06baa-2d37-41e5-ada7-e44389198214 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:27.764831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-10a06baa-2d37-41e5-ada7-e44389198214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:27.940013 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:27.952191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a06baa-2d37-41e5-ada7-e44389198214 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:28.077529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ab79b-703f-440b-9d3c-a19a416b1dd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:28.078185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ab79b-703f-440b-9d3c-a19a416b1dd8 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:28.078464 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ab79b-703f-440b-9d3c-a19a416b1dd8 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:28.078743 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ab79b-703f-440b-9d3c-a19a416b1dd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:28.080401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-340ab79b-703f-440b-9d3c-a19a416b1dd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:28.180917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ab79b-703f-440b-9d3c-a19a416b1dd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:28.183056 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ab79b-703f-440b-9d3c-a19a416b1dd8 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:28.194384 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:28.219147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 687/1382] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:30.304203 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efcce1-f084-46f1-94e5-08dfba917c81 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:30.304913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efcce1-f084-46f1-94e5-08dfba917c81 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:30.304913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efcce1-f084-46f1-94e5-08dfba917c81 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:30.306222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 696/1383] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:30.312670 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6b3d2-cacf-4dbd-8467-eaf709d3401d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:30.313304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6b3d2-cacf-4dbd-8467-eaf709d3401d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:30.313304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6b3d2-cacf-4dbd-8467-eaf709d3401d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:30.395678 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4cd6b3d2-cacf-4dbd-8467-eaf709d3401d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:30.397490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4cd6b3d2-cacf-4dbd-8467-eaf709d3401d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:30.517030 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 688/1384] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:30.562931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efcce1-f084-46f1-94e5-08dfba917c81 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:30.564551 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f2efcce1-f084-46f1-94e5-08dfba917c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:30.658856 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:30.734857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efcce1-f084-46f1-94e5-08dfba917c81 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:30.880132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeda164-16eb-4c07-a6d3-a9950d469b9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:30.880605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeda164-16eb-4c07-a6d3-a9950d469b9f service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:30.880956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeda164-16eb-4c07-a6d3-a9950d469b9f service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:30.881312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeda164-16eb-4c07-a6d3-a9950d469b9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:30.883962 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-0aeda164-16eb-4c07-a6d3-a9950d469b9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:30.948766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:31.002763 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeda164-16eb-4c07-a6d3-a9950d469b9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:31.005122 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeda164-16eb-4c07-a6d3-a9950d469b9f service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:31.030170 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 697/1385] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:33.150648 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd272389-8ce1-467d-b1da-3da39f2960f0 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:33.151364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd272389-8ce1-467d-b1da-3da39f2960f0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:33.151364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd272389-8ce1-467d-b1da-3da39f2960f0 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:33.152696 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 689/1386] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:16:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:33.158749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe4e4d-a889-4204-97c3-fac760e3a0e1 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:33.159268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe4e4d-a889-4204-97c3-fac760e3a0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:33.159436 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe4e4d-a889-4204-97c3-fac760e3a0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:33.240772 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-01fe4e4d-a889-4204-97c3-fac760e3a0e1 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:33.242116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-01fe4e4d-a889-4204-97c3-fac760e3a0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:33.347239 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 698/1387] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:33.395001 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd272389-8ce1-467d-b1da-3da39f2960f0 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:33.396661 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cd272389-8ce1-467d-b1da-3da39f2960f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:33.483632 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:33.573794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd272389-8ce1-467d-b1da-3da39f2960f0 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:33.717535 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eabfe0-bc14-42fd-b416-b4046b5f2284 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:33.718228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eabfe0-bc14-42fd-b416-b4046b5f2284 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:33.718517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eabfe0-bc14-42fd-b416-b4046b5f2284 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:33.718798 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eabfe0-bc14-42fd-b416-b4046b5f2284 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:33.720598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f7eabfe0-bc14-42fd-b416-b4046b5f2284 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:33.764706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:33.825038 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eabfe0-bc14-42fd-b416-b4046b5f2284 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:33.826813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eabfe0-bc14-42fd-b416-b4046b5f2284 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:33.846614 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 690/1388] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:35.965735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407562d5-6026-4e48-9016-d1c963d2e9f4 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:35.966295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407562d5-6026-4e48-9016-d1c963d2e9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:35.966295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407562d5-6026-4e48-9016-d1c963d2e9f4 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:35.967638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 699/1389] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:35.973181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f905a-d240-4deb-b33e-c90f34ecbbe7 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:35.973631 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f905a-d240-4deb-b33e-c90f34ecbbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:35.973742 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f905a-d240-4deb-b33e-c90f34ecbbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:36.044610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-af0f905a-d240-4deb-b33e-c90f34ecbbe7 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:36.045817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-af0f905a-d240-4deb-b33e-c90f34ecbbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:36.145431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 691/1390] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:36.190798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407562d5-6026-4e48-9016-d1c963d2e9f4 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:36.192333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-407562d5-6026-4e48-9016-d1c963d2e9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:36.225815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:36.350432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407562d5-6026-4e48-9016-d1c963d2e9f4 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:36.453925 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:36.466810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b46537-3be7-433d-9126-95b352acf583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:36.467444 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b46537-3be7-433d-9126-95b352acf583 service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:36.467720 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b46537-3be7-433d-9126-95b352acf583 service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:36.467978 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b46537-3be7-433d-9126-95b352acf583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:36.469617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f9b46537-3be7-433d-9126-95b352acf583 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:36.562830 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b46537-3be7-433d-9126-95b352acf583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:36.564489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b46537-3be7-433d-9126-95b352acf583 service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:36.583468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 700/1391] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:37.077769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58eae9-deb2-4d3e-ba3a-8388e491ec7c None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:37.078368 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58eae9-deb2-4d3e-ba3a-8388e491ec7c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:37.078368 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58eae9-deb2-4d3e-ba3a-8388e491ec7c None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:37.079776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 692/1392] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:37.085331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477076cb-7300-40f6-963c-9ac4203414e7 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:37.085762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477076cb-7300-40f6-963c-9ac4203414e7 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:37.085971 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477076cb-7300-40f6-963c-9ac4203414e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:37.155410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-477076cb-7300-40f6-963c-9ac4203414e7 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:37.157409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-477076cb-7300-40f6-963c-9ac4203414e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:37.241345 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:37.282771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 701/1393] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:16:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:37.342170 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a85d3c-e99c-4d58-8877-19e2952318f9 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:37.342407 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a85d3c-e99c-4d58-8877-19e2952318f9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:37.342716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a85d3c-e99c-4d58-8877-19e2952318f9 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:37.344242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 693/1394] 10.222.0.48 () {54 vars in 966 bytes} [Mon Aug 1 15:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:37.350044 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f950a6-a13d-49f0-8966-34e0cf598885 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:37.350311 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f950a6-a13d-49f0-8966-34e0cf598885 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:37.350617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f950a6-a13d-49f0-8966-34e0cf598885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:37.434275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-b3f950a6-a13d-49f0-8966-34e0cf598885 None None] MFA Rules not processed for user `94f696dfe4b34c4aa19104b5f7ee6e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:37.435462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-b3f950a6-a13d-49f0-8966-34e0cf598885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:37.516105 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 702/1395] 10.222.0.48 () {58 vars in 1063 bytes} [Mon Aug 1 15:16:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:37.523916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a85d3c-e99c-4d58-8877-19e2952318f9 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:37.525605 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-64a85d3c-e99c-4d58-8877-19e2952318f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:37.540578 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:37.635101 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a85d3c-e99c-4d58-8877-19e2952318f9 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:37.714717 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81d2e9-37f9-46e3-a948-91e46568cdac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f696dfe4b34c4aa19104b5f7ee6e70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:37.715618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81d2e9-37f9-46e3-a948-91e46568cdac service glance] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:37.715892 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81d2e9-37f9-46e3-a948-91e46568cdac service glance] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:37.716161 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81d2e9-37f9-46e3-a948-91e46568cdac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:37.717903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3b81d2e9-37f9-46e3-a948-91e46568cdac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:37.791587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:37.834650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81d2e9-37f9-46e3-a948-91e46568cdac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:37.836129 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81d2e9-37f9-46e3-a948-91e46568cdac service glance] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:37.854559 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 694/1396] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:38.619603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422fa27-e29e-4c6f-b27a-c6019472fc6b None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:38.620188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422fa27-e29e-4c6f-b27a-c6019472fc6b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:38.620384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422fa27-e29e-4c6f-b27a-c6019472fc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:38.628060 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedd34f-8e57-4207-8832-2c1d1b2e7720 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:38.628528 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedd34f-8e57-4207-8832-2c1d1b2e7720 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:38.628658 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aedd34f-8e57-4207-8832-2c1d1b2e7720 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:38.630075 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 695/1397] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:16:38.634936 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c738fe7-c168-46e2-9317-05b3793077c7 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:38.635228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c738fe7-c168-46e2-9317-05b3793077c7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:38.635630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c738fe7-c168-46e2-9317-05b3793077c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:38.689721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-e422fa27-e29e-4c6f-b27a-c6019472fc6b None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:38.691000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e422fa27-e29e-4c6f-b27a-c6019472fc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:38.699810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:38.704414 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-9c738fe7-c168-46e2-9317-05b3793077c7 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:38.705615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9c738fe7-c168-46e2-9317-05b3793077c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:38.781043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 703/1398] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 15:16:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:38.808064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 696/1399] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:38.829834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23404f-a2f3-471d-99ca-80d8f294da84 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:38.830115 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23404f-a2f3-471d-99ca-80d8f294da84 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:38.830382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23404f-a2f3-471d-99ca-80d8f294da84 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:38.831762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 704/1400] 10.222.0.48 () {54 vars in 943 bytes} [Mon Aug 1 15:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:38.838218 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f789eb0-233a-4900-bc53-b0e2ab5b4234 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:38.838506 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f789eb0-233a-4900-bc53-b0e2ab5b4234 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:38.838778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f789eb0-233a-4900-bc53-b0e2ab5b4234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:38.852817 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23404f-a2f3-471d-99ca-80d8f294da84 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:38.854382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4c23404f-a2f3-471d-99ca-80d8f294da84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:38.869762 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:38.925711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4f789eb0-233a-4900-bc53-b0e2ab5b4234 None None] MFA Rules not processed for user `25a26f54accf4264aef646c3757de7e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:38.926962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4f789eb0-233a-4900-bc53-b0e2ab5b4234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:39.008741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23404f-a2f3-471d-99ca-80d8f294da84 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:39.012437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:39.036335 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 697/1401] 10.222.0.48 () {58 vars in 1040 bytes} [Mon Aug 1 15:16:38 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:39.046370 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f789eb0-233a-4900-bc53-b0e2ab5b4234 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:39.056508 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f789eb0-233a-4900-bc53-b0e2ab5b4234 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:39.130188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a45fe-32cc-41dc-a8f6-bcb7e4b62b06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:39.130750 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a45fe-32cc-41dc-a8f6-bcb7e4b62b06 service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:39.131025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a45fe-32cc-41dc-a8f6-bcb7e4b62b06 service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:39.131299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a45fe-32cc-41dc-a8f6-bcb7e4b62b06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:39.131918 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:39.133342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f55a45fe-32cc-41dc-a8f6-bcb7e4b62b06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:39.167577 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aad1f6-e451-40e0-a68e-4c4d06730e3e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:39.168312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aad1f6-e451-40e0-a68e-4c4d06730e3e service swift] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:39.168624 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aad1f6-e451-40e0-a68e-4c4d06730e3e service swift] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:39.168899 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aad1f6-e451-40e0-a68e-4c4d06730e3e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:39.170725 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c0aad1f6-e451-40e0-a68e-4c4d06730e3e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:39.227594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a45fe-32cc-41dc-a8f6-bcb7e4b62b06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:39.229201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a45fe-32cc-41dc-a8f6-bcb7e4b62b06 service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:39.248511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 705/1402] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:39.257317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aad1f6-e451-40e0-a68e-4c4d06730e3e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:39.259016 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aad1f6-e451-40e0-a68e-4c4d06730e3e service swift] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:39.268257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 698/1403] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:16:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:16:41.326292 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4a075-5fcf-43e7-b5bb-26e764726d94 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:41.326860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4a075-5fcf-43e7-b5bb-26e764726d94 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:41.327077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4a075-5fcf-43e7-b5bb-26e764726d94 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:41.328590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 706/1404] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:16:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:41.334464 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1934ee-7faf-4f08-a456-3ed36f8f0770 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:41.334964 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1934ee-7faf-4f08-a456-3ed36f8f0770 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:41.335058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1934ee-7faf-4f08-a456-3ed36f8f0770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:41.454027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-9c1934ee-7faf-4f08-a456-3ed36f8f0770 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:41.460673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9c1934ee-7faf-4f08-a456-3ed36f8f0770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:41.534512 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:41.593906 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 699/1405] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:41.637456 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4a075-5fcf-43e7-b5bb-26e764726d94 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:41.639064 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-ace4a075-5fcf-43e7-b5bb-26e764726d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:41.651295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:41.822140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4a075-5fcf-43e7-b5bb-26e764726d94 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:41.913113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:41.951344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef19489-a68e-44f9-8b75-9e58c2ba7287 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:41.952328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef19489-a68e-44f9-8b75-9e58c2ba7287 service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:41.952604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef19489-a68e-44f9-8b75-9e58c2ba7287 service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:41.952878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef19489-a68e-44f9-8b75-9e58c2ba7287 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:41.954641 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-bef19489-a68e-44f9-8b75-9e58c2ba7287 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:42.059630 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef19489-a68e-44f9-8b75-9e58c2ba7287 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:42.061413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef19489-a68e-44f9-8b75-9e58c2ba7287 service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:42.080092 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 707/1406] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:44.194752 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503291a6-4cea-4f77-8948-d7a943d17745 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.195382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503291a6-4cea-4f77-8948-d7a943d17745 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.195382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503291a6-4cea-4f77-8948-d7a943d17745 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.200981 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 700/1407] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:16:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:16:44.202731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5212c-50f5-4410-914b-56d70e3d1d66 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.203577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5212c-50f5-4410-914b-56d70e3d1d66 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.204080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5212c-50f5-4410-914b-56d70e3d1d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.279472 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-4df5212c-50f5-4410-914b-56d70e3d1d66 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:44.280843 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4df5212c-50f5-4410-914b-56d70e3d1d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:44.332898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:44.398636 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 708/1408] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:44.449418 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503291a6-4cea-4f77-8948-d7a943d17745 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:44.451042 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-503291a6-4cea-4f77-8948-d7a943d17745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:44.633966 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503291a6-4cea-4f77-8948-d7a943d17745 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:44.681898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:44.763889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93596707-1423-4489-a2d5-08df5380e029 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:44.764556 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93596707-1423-4489-a2d5-08df5380e029 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.764916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93596707-1423-4489-a2d5-08df5380e029 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.765248 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93596707-1423-4489-a2d5-08df5380e029 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.767070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-93596707-1423-4489-a2d5-08df5380e029 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:44.874269 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93596707-1423-4489-a2d5-08df5380e029 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:44.876074 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93596707-1423-4489-a2d5-08df5380e029 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:44.899539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 701/1409] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:16:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:16:54.396770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=23 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:54.897267 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=79 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:02.305251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ed5d5-f0cd-4ed3-955a-a01fa78cd7eb None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:02.305864 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ed5d5-f0cd-4ed3-955a-a01fa78cd7eb None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:02.305864 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ed5d5-f0cd-4ed3-955a-a01fa78cd7eb None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:02.307332 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 709/1410] 10.222.0.48 () {54 vars in 918 bytes} [Mon Aug 1 15:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:02.312741 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32890efb-d6dc-4429-9d08-eb67b4a4c5b4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:02.313202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32890efb-d6dc-4429-9d08-eb67b4a4c5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:02.313306 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32890efb-d6dc-4429-9d08-eb67b4a4c5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:02.398533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-32890efb-d6dc-4429-9d08-eb67b4a4c5b4 None None] MFA Rules not processed for user `098d8d7e35804ccc931ddfba07f688a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:02.399764 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-32890efb-d6dc-4429-9d08-eb67b4a4c5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:02.516776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 702/1411] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Aug 1 15:17:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:02.536286 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ee14f-2c1b-4388-9744-71435df467d8 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:02.536566 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ee14f-2c1b-4388-9744-71435df467d8 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:02.536834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ee14f-2c1b-4388-9744-71435df467d8 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:02.538263 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 710/1412] 10.222.0.48 () {54 vars in 918 bytes} [Mon Aug 1 15:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:02.544793 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1ca18-e3e1-4b40-89cd-59014b5490c4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:02.545074 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1ca18-e3e1-4b40-89cd-59014b5490c4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:02.545355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a1ca18-e3e1-4b40-89cd-59014b5490c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:02.621265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:02.625078 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f1a1ca18-e3e1-4b40-89cd-59014b5490c4 None None] MFA Rules not processed for user `2de0ff99a2b74979a28a8b10e9ebdfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:02.626710 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f1a1ca18-e3e1-4b40-89cd-59014b5490c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:02.748064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 703/1413] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Aug 1 15:17:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:17:05.260535 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd2c78-37c7-45a7-a4d0-6b7644b5670e None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:05.261099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd2c78-37c7-45a7-a4d0-6b7644b5670e None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:05.261099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd2c78-37c7-45a7-a4d0-6b7644b5670e None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:05.262916 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:17:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:05.268862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb48c6-3c12-4448-b9e3-2c43e46c0cd0 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:05.269290 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb48c6-3c12-4448-b9e3-2c43e46c0cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:05.269411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb48c6-3c12-4448-b9e3-2c43e46c0cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:05.345519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-10bb48c6-3c12-4448-b9e3-2c43e46c0cd0 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:05.346774 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-10bb48c6-3c12-4448-b9e3-2c43e46c0cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:05.408408 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:05.459127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:17:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:05.503129 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd2c78-37c7-45a7-a4d0-6b7644b5670e None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:05.504669 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-fdfd2c78-37c7-45a7-a4d0-6b7644b5670e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:05.675814 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd2c78-37c7-45a7-a4d0-6b7644b5670e None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:05.757232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:05.800413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d136f7b-0e35-442f-9ab1-34a73ad9e314 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:05.801027 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d136f7b-0e35-442f-9ab1-34a73ad9e314 service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:05.801321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d136f7b-0e35-442f-9ab1-34a73ad9e314 service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:05.801594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d136f7b-0e35-442f-9ab1-34a73ad9e314 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:05.803316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1d136f7b-0e35-442f-9ab1-34a73ad9e314 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:05.900921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d136f7b-0e35-442f-9ab1-34a73ad9e314 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:05.902599 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d136f7b-0e35-442f-9ab1-34a73ad9e314 service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:05.924543 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 712/1416] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:06.841263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dde711-a5a0-44b4-bcff-82240085413a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:06.841879 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dde711-a5a0-44b4-bcff-82240085413a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:06.841879 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dde711-a5a0-44b4-bcff-82240085413a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:06.843926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:17:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:06.852826 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf48cd-042f-4234-95fb-fab70136c2ff None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:06.853232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf48cd-042f-4234-95fb-fab70136c2ff None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:06.853719 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf48cd-042f-4234-95fb-fab70136c2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:06.955334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2adf48cd-042f-4234-95fb-fab70136c2ff None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:06.956584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2adf48cd-042f-4234-95fb-fab70136c2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:06.988288 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:07.080293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:07.242749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dde711-a5a0-44b4-bcff-82240085413a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:07.244293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d8dde711-a5a0-44b4-bcff-82240085413a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:07.417696 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dde711-a5a0-44b4-bcff-82240085413a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:07.466156 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:07.545948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd3515-b18e-4bd5-930a-b5ef8c93ce54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:07.546832 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd3515-b18e-4bd5-930a-b5ef8c93ce54 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:07.547243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd3515-b18e-4bd5-930a-b5ef8c93ce54 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:07.547659 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd3515-b18e-4bd5-930a-b5ef8c93ce54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:07.550433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-88fd3515-b18e-4bd5-930a-b5ef8c93ce54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:07.644890 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fd3515-b18e-4bd5-930a-b5ef8c93ce54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:07.646567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fd3515-b18e-4bd5-930a-b5ef8c93ce54 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:07.665167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 706/1419] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:08.587609 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecae44f-c619-4651-ae02-ed549a4e202a None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:08.588183 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecae44f-c619-4651-ae02-ed549a4e202a None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:08.588183 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecae44f-c619-4651-ae02-ed549a4e202a None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:08.589749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:17:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:08.595789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18661ff1-86c3-42b7-95b9-831486d213ac None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:08.596444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18661ff1-86c3-42b7-95b9-831486d213ac None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:08.596444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18661ff1-86c3-42b7-95b9-831486d213ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:08.663243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:08.691811 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-18661ff1-86c3-42b7-95b9-831486d213ac None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:08.693082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-18661ff1-86c3-42b7-95b9-831486d213ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:08.819105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:08.975232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecae44f-c619-4651-ae02-ed549a4e202a None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:08.976821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6ecae44f-c619-4651-ae02-ed549a4e202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:09.138851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecae44f-c619-4651-ae02-ed549a4e202a None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:09.152526 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:09.261603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e3455-a984-4769-b7cb-c60138c2fdb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:09.262380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e3455-a984-4769-b7cb-c60138c2fdb8 service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:09.262650 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e3455-a984-4769-b7cb-c60138c2fdb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:09.262921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e3455-a984-4769-b7cb-c60138c2fdb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:09.264615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e54e3455-a984-4769-b7cb-c60138c2fdb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:09.372784 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e3455-a984-4769-b7cb-c60138c2fdb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:09.374436 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e3455-a984-4769-b7cb-c60138c2fdb8 service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:09.393763 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 715/1422] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:17:10.275483 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f071fbc-9b72-4ce7-b1e5-96cd1cdf7b55 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:10.276246 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f071fbc-9b72-4ce7-b1e5-96cd1cdf7b55 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:10.276246 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f071fbc-9b72-4ce7-b1e5-96cd1cdf7b55 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:10.278084 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:17:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:10.283806 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ef5f3-e04f-41f9-968a-05f1c7d65302 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:10.284272 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ef5f3-e04f-41f9-968a-05f1c7d65302 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:10.284362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ef5f3-e04f-41f9-968a-05f1c7d65302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:10.306902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:10.382625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-991ef5f3-e04f-41f9-968a-05f1c7d65302 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:10.383885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-991ef5f3-e04f-41f9-968a-05f1c7d65302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:10.505522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:10.666854 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f071fbc-9b72-4ce7-b1e5-96cd1cdf7b55 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:10.668485 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0f071fbc-9b72-4ce7-b1e5-96cd1cdf7b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:10.799442 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:10.840243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f071fbc-9b72-4ce7-b1e5-96cd1cdf7b55 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:10.965138 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30557c7a-0c9a-4c25-8fd1-18560af28c1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:10.965672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30557c7a-0c9a-4c25-8fd1-18560af28c1e service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:10.965976 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30557c7a-0c9a-4c25-8fd1-18560af28c1e service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:10.966250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30557c7a-0c9a-4c25-8fd1-18560af28c1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:10.967898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-30557c7a-0c9a-4c25-8fd1-18560af28c1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:11.051501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:11.067833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30557c7a-0c9a-4c25-8fd1-18560af28c1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:11.069498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30557c7a-0c9a-4c25-8fd1-18560af28c1e service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:11.088557 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 709/1425] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:11.970812 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731a928-e2cb-4305-99af-13bda827f3fe None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:11.971637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731a928-e2cb-4305-99af-13bda827f3fe None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:11.971789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731a928-e2cb-4305-99af-13bda827f3fe None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:11.974777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:17:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:11.980432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fd733-4f13-40ff-aed2-dbb836008738 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:11.980895 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fd733-4f13-40ff-aed2-dbb836008738 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:11.981011 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fd733-4f13-40ff-aed2-dbb836008738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:12.067712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7b5fd733-4f13-40ff-aed2-dbb836008738 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:12.068941 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7b5fd733-4f13-40ff-aed2-dbb836008738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:12.191004 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:17:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:12.343595 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731a928-e2cb-4305-99af-13bda827f3fe None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:12.345118 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2731a928-e2cb-4305-99af-13bda827f3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:12.440059 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:12.519672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731a928-e2cb-4305-99af-13bda827f3fe None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:12.657103 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce27147-b033-4b4f-9c0a-4e81c6d3d544 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:12.657644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce27147-b033-4b4f-9c0a-4e81c6d3d544 service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:12.657972 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce27147-b033-4b4f-9c0a-4e81c6d3d544 service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:12.658239 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce27147-b033-4b4f-9c0a-4e81c6d3d544 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:12.659922 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-0ce27147-b033-4b4f-9c0a-4e81c6d3d544 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:12.712510 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:12.760455 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce27147-b033-4b4f-9c0a-4e81c6d3d544 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:12.762117 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce27147-b033-4b4f-9c0a-4e81c6d3d544 service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:12.781319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 718/1428] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:13.593408 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c46bd-86f2-4e11-811b-4b9e4dbaf721 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:13.594369 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c46bd-86f2-4e11-811b-4b9e4dbaf721 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:13.594369 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c46bd-86f2-4e11-811b-4b9e4dbaf721 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:13.596516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:17:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:13.602146 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebea629-53ee-474a-9e53-e0317a96fabd None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:13.602591 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebea629-53ee-474a-9e53-e0317a96fabd None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:13.602699 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebea629-53ee-474a-9e53-e0317a96fabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:13.696060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-6ebea629-53ee-474a-9e53-e0317a96fabd None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:13.697305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6ebea629-53ee-474a-9e53-e0317a96fabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:13.809282 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:13.974507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c46bd-86f2-4e11-811b-4b9e4dbaf721 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:13.976185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-940c46bd-86f2-4e11-811b-4b9e4dbaf721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:14.045363 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:14.153844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c46bd-86f2-4e11-811b-4b9e4dbaf721 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:14.278560 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc2ca8-1248-4057-888b-6a21f3c77ccb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:14.279185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc2ca8-1248-4057-888b-6a21f3c77ccb service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:14.279458 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc2ca8-1248-4057-888b-6a21f3c77ccb service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:14.279752 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc2ca8-1248-4057-888b-6a21f3c77ccb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:14.281391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3dcc2ca8-1248-4057-888b-6a21f3c77ccb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:14.288436 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:14.373815 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc2ca8-1248-4057-888b-6a21f3c77ccb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:14.375448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc2ca8-1248-4057-888b-6a21f3c77ccb service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:14.393489 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 712/1431] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:15.220800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c16b6-61c7-40e2-b1bb-238f7a72048e None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:15.221745 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c16b6-61c7-40e2-b1bb-238f7a72048e None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:15.221745 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c16b6-61c7-40e2-b1bb-238f7a72048e None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:15.224107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:17:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:15.232420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f1bbb-2baa-47e8-8077-3cde9b2e30d0 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:15.233180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f1bbb-2baa-47e8-8077-3cde9b2e30d0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:15.233180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f1bbb-2baa-47e8-8077-3cde9b2e30d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:15.325571 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7f5f1bbb-2baa-47e8-8077-3cde9b2e30d0 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:15.326860 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7f5f1bbb-2baa-47e8-8077-3cde9b2e30d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:15.412964 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:15.443699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:15.594209 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c16b6-61c7-40e2-b1bb-238f7a72048e None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:15.596335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-032c16b6-61c7-40e2-b1bb-238f7a72048e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:15.621958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:15.771732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c16b6-61c7-40e2-b1bb-238f7a72048e None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:15.862370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:15.889664 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28565387-1aaa-4bc3-bd51-a266d2474263 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:15.890318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28565387-1aaa-4bc3-bd51-a266d2474263 service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:15.890599 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28565387-1aaa-4bc3-bd51-a266d2474263 service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:15.890865 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28565387-1aaa-4bc3-bd51-a266d2474263 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:15.892497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-28565387-1aaa-4bc3-bd51-a266d2474263 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:15.997213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28565387-1aaa-4bc3-bd51-a266d2474263 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:15.998875 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28565387-1aaa-4bc3-bd51-a266d2474263 service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:16.024587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 721/1434] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:16.858405 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca2980-0073-488a-b9b9-f558a1b30333 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:16.858960 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca2980-0073-488a-b9b9-f558a1b30333 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:16.858960 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca2980-0073-488a-b9b9-f558a1b30333 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:16.860383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:16.865516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d788ab1-03fc-4bde-9737-a4c2b0ba41d1 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:16.866008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d788ab1-03fc-4bde-9737-a4c2b0ba41d1 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:16.866114 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d788ab1-03fc-4bde-9737-a4c2b0ba41d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:16.962346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-6d788ab1-03fc-4bde-9737-a4c2b0ba41d1 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:16.963583 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6d788ab1-03fc-4bde-9737-a4c2b0ba41d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:17.010647 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:17.071416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:17.225719 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca2980-0073-488a-b9b9-f558a1b30333 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:17.227307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-47ca2980-0073-488a-b9b9-f558a1b30333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:17.388350 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca2980-0073-488a-b9b9-f558a1b30333 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:17.445905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:17.513007 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d758f6-fe20-48ad-a963-2f6cf3fef133 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:17.513643 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d758f6-fe20-48ad-a963-2f6cf3fef133 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:17.513960 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d758f6-fe20-48ad-a963-2f6cf3fef133 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:17.514239 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d758f6-fe20-48ad-a963-2f6cf3fef133 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:17.515920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b4d758f6-fe20-48ad-a963-2f6cf3fef133 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:17.619185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d758f6-fe20-48ad-a963-2f6cf3fef133 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:17.620990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d758f6-fe20-48ad-a963-2f6cf3fef133 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:17.641541 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 715/1437] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:18.469730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30ebb0-ae45-4553-91a1-c5e8fa6140ac None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:18.470665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30ebb0-ae45-4553-91a1-c5e8fa6140ac None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:18.470665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30ebb0-ae45-4553-91a1-c5e8fa6140ac None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:18.472798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:17:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:18.481962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de44932-4fba-4724-ba5b-fac08082dfd1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:18.482396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de44932-4fba-4724-ba5b-fac08082dfd1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:18.482571 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de44932-4fba-4724-ba5b-fac08082dfd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:18.561813 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7de44932-4fba-4724-ba5b-fac08082dfd1 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:18.563070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7de44932-4fba-4724-ba5b-fac08082dfd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:18.565165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:18.673232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:18.836396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30ebb0-ae45-4553-91a1-c5e8fa6140ac None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:18.838034 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6f30ebb0-ae45-4553-91a1-c5e8fa6140ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:18.995283 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30ebb0-ae45-4553-91a1-c5e8fa6140ac None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:19.026152 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:19.128827 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3105c-c184-4b75-9245-70791f5987b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:19.129361 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3105c-c184-4b75-9245-70791f5987b3 service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:19.129637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3105c-c184-4b75-9245-70791f5987b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:19.129939 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3105c-c184-4b75-9245-70791f5987b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:19.131655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-eed3105c-c184-4b75-9245-70791f5987b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:19.227265 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3105c-c184-4b75-9245-70791f5987b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:19.228899 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3105c-c184-4b75-9245-70791f5987b3 service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:19.248164 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 724/1440] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:21.170357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dca99-016f-4dd4-83bb-5d187c1bd6c3 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:21.171258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dca99-016f-4dd4-83bb-5d187c1bd6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:21.171258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dca99-016f-4dd4-83bb-5d187c1bd6c3 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:21.173432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:17:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:21.182713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a962a8b-b060-4423-81da-4969007c99fc None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:21.183489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a962a8b-b060-4423-81da-4969007c99fc None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:21.183682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a962a8b-b060-4423-81da-4969007c99fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:21.239237 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:21.268593 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2a962a8b-b060-4423-81da-4969007c99fc None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:21.269918 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2a962a8b-b060-4423-81da-4969007c99fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:21.377318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:21.422035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83dca99-016f-4dd4-83bb-5d187c1bd6c3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:21.423543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b83dca99-016f-4dd4-83bb-5d187c1bd6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:21.537021 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83dca99-016f-4dd4-83bb-5d187c1bd6c3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:21.573399 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:21.621105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d62415-25b7-412d-af44-eec9006fcab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:21.621739 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d62415-25b7-412d-af44-eec9006fcab4 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:21.622161 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d62415-25b7-412d-af44-eec9006fcab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:21.622426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d62415-25b7-412d-af44-eec9006fcab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:21.624089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b8d62415-25b7-412d-af44-eec9006fcab4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:21.740989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d62415-25b7-412d-af44-eec9006fcab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:21.742506 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d62415-25b7-412d-af44-eec9006fcab4 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:21.762459 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 718/1443] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:23.927688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb1a15-5e9b-407f-9427-15f0d04b3888 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:23.928408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb1a15-5e9b-407f-9427-15f0d04b3888 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:23.928408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb1a15-5e9b-407f-9427-15f0d04b3888 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:23.930173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:17:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:23.935985 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9c58c-fafa-49f9-9b22-89902213662b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:23.936437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9c58c-fafa-49f9-9b22-89902213662b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:23.936640 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9c58c-fafa-49f9-9b22-89902213662b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:24.015472 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-8eb9c58c-fafa-49f9-9b22-89902213662b None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:24.016618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8eb9c58c-fafa-49f9-9b22-89902213662b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:24.034457 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:24.134678 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:24.179730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb1a15-5e9b-407f-9427-15f0d04b3888 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:24.181295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1ddb1a15-5e9b-407f-9427-15f0d04b3888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:24.300830 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb1a15-5e9b-407f-9427-15f0d04b3888 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:24.316837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:24.385200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cb42e-9acc-4302-83d2-579989fd1da3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:24.385893 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cb42e-9acc-4302-83d2-579989fd1da3 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:24.386143 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cb42e-9acc-4302-83d2-579989fd1da3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:24.386420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cb42e-9acc-4302-83d2-579989fd1da3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:24.388277 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4c6cb42e-9acc-4302-83d2-579989fd1da3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:24.513111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6cb42e-9acc-4302-83d2-579989fd1da3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:24.515403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6cb42e-9acc-4302-83d2-579989fd1da3 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:24.546264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 727/1446] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:26.876737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bca04-e1d3-444a-8f06-66d1b15fd96a None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:26.877358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bca04-e1d3-444a-8f06-66d1b15fd96a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:26.877358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988bca04-e1d3-444a-8f06-66d1b15fd96a None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:26.878685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:26.884936 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a165f3a-1884-40fd-b6ea-3d3b9a2c2c43 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:26.885356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a165f3a-1884-40fd-b6ea-3d3b9a2c2c43 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:26.885479 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a165f3a-1884-40fd-b6ea-3d3b9a2c2c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:26.944166 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:26.964770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-5a165f3a-1884-40fd-b6ea-3d3b9a2c2c43 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:26.966102 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5a165f3a-1884-40fd-b6ea-3d3b9a2c2c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:27.074532 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:27.116261 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bca04-e1d3-444a-8f06-66d1b15fd96a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:27.117881 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-988bca04-e1d3-444a-8f06-66d1b15fd96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:27.241436 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988bca04-e1d3-444a-8f06-66d1b15fd96a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:27.287929 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:27.327553 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17086f90-9ef9-4e84-8759-f1776c93d352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:27.328160 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17086f90-9ef9-4e84-8759-f1776c93d352 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:27.328438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17086f90-9ef9-4e84-8759-f1776c93d352 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:27.328710 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17086f90-9ef9-4e84-8759-f1776c93d352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:27.330381 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-17086f90-9ef9-4e84-8759-f1776c93d352 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:27.470081 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17086f90-9ef9-4e84-8759-f1776c93d352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:27.471589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17086f90-9ef9-4e84-8759-f1776c93d352 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:27.491311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 721/1449] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:29.669394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b0227-0405-47e5-b628-c5895f604868 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:29.670022 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b0227-0405-47e5-b628-c5895f604868 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:29.670022 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b0227-0405-47e5-b628-c5895f604868 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:29.671351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:29.677904 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52fada-a901-4315-a308-c7ef95b2987d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:29.678695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52fada-a901-4315-a308-c7ef95b2987d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:29.678849 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52fada-a901-4315-a308-c7ef95b2987d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:29.757993 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-af52fada-a901-4315-a308-c7ef95b2987d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:29.759228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-af52fada-a901-4315-a308-c7ef95b2987d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:29.783354 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:29.888320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:29.927449 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b0227-0405-47e5-b628-c5895f604868 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:29.928998 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-ce1b0227-0405-47e5-b628-c5895f604868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:30.047730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b0227-0405-47e5-b628-c5895f604868 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:30.068484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:30.128238 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc6515-8186-4f8c-b250-12f2bcb02046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:30.128891 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc6515-8186-4f8c-b250-12f2bcb02046 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:30.129166 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc6515-8186-4f8c-b250-12f2bcb02046 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:30.129435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc6515-8186-4f8c-b250-12f2bcb02046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:30.131137 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1acc6515-8186-4f8c-b250-12f2bcb02046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:30.266702 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc6515-8186-4f8c-b250-12f2bcb02046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:30.268041 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc6515-8186-4f8c-b250-12f2bcb02046 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:30.288899 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 730/1452] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:32.391631 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89890674-c18b-4fc9-9673-e0aba60d7ed1 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:32.392516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89890674-c18b-4fc9-9673-e0aba60d7ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:32.392516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89890674-c18b-4fc9-9673-e0aba60d7ed1 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:32.394862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:17:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:32.403939 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27540d90-b79c-455c-83c3-0e701af43f5a None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:32.404705 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27540d90-b79c-455c-83c3-0e701af43f5a None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:32.404908 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27540d90-b79c-455c-83c3-0e701af43f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:32.480981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:32.486018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-27540d90-b79c-455c-83c3-0e701af43f5a None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:32.489555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-27540d90-b79c-455c-83c3-0e701af43f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:32.608635 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:32.651893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89890674-c18b-4fc9-9673-e0aba60d7ed1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:32.653765 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-89890674-c18b-4fc9-9673-e0aba60d7ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:32.776045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89890674-c18b-4fc9-9673-e0aba60d7ed1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:32.837214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:32.864761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840742b7-708d-4ed3-8c62-3de62b2d0b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:32.865386 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840742b7-708d-4ed3-8c62-3de62b2d0b51 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:32.865670 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840742b7-708d-4ed3-8c62-3de62b2d0b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:32.865971 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840742b7-708d-4ed3-8c62-3de62b2d0b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:32.867642 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-840742b7-708d-4ed3-8c62-3de62b2d0b51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:32.988843 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840742b7-708d-4ed3-8c62-3de62b2d0b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:32.990267 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840742b7-708d-4ed3-8c62-3de62b2d0b51 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:33.008412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 724/1455] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:35.059017 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a926ac5-414d-4651-904c-550fd1ccedfb None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:35.059595 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a926ac5-414d-4651-904c-550fd1ccedfb None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:35.059595 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a926ac5-414d-4651-904c-550fd1ccedfb None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:35.060915 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:35.067276 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf3247-d5f9-4049-aa3c-801de00bc82f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:35.067703 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf3247-d5f9-4049-aa3c-801de00bc82f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:35.067825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf3247-d5f9-4049-aa3c-801de00bc82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:35.146635 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-1edf3247-d5f9-4049-aa3c-801de00bc82f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:35.147794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1edf3247-d5f9-4049-aa3c-801de00bc82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:35.181060 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:35.257494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:35.303510 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a926ac5-414d-4651-904c-550fd1ccedfb None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:35.305194 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2a926ac5-414d-4651-904c-550fd1ccedfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:35.416363 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a926ac5-414d-4651-904c-550fd1ccedfb None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:35.447548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:35.505674 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0a00f-1710-4522-add8-38ebbebcab71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:35.506414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0a00f-1710-4522-add8-38ebbebcab71 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:35.506693 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0a00f-1710-4522-add8-38ebbebcab71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:35.506961 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0a00f-1710-4522-add8-38ebbebcab71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:35.508628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-93f0a00f-1710-4522-add8-38ebbebcab71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:35.633296 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0a00f-1710-4522-add8-38ebbebcab71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:35.634780 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0a00f-1710-4522-add8-38ebbebcab71 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:35.655191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 733/1458] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:37.738176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b0960-338c-4f2d-a93f-1c8475671578 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:37.738914 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b0960-338c-4f2d-a93f-1c8475671578 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:37.739122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b0960-338c-4f2d-a93f-1c8475671578 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:37.741327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:17:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:37.746978 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fb306-8208-4166-83fe-4dc5501d21a8 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:37.747407 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fb306-8208-4166-83fe-4dc5501d21a8 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:37.747502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fb306-8208-4166-83fe-4dc5501d21a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:37.831882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-360fb306-8208-4166-83fe-4dc5501d21a8 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:37.833167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-360fb306-8208-4166-83fe-4dc5501d21a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:37.842158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:37.945692 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:37.989974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b0960-338c-4f2d-a93f-1c8475671578 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:37.992424 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3f3b0960-338c-4f2d-a93f-1c8475671578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:38.111624 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b0960-338c-4f2d-a93f-1c8475671578 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:38.168984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:38.194585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5baf8e2-ed95-4828-9828-c9715e22b66e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:38.195216 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5baf8e2-ed95-4828-9828-c9715e22b66e service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:38.195497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5baf8e2-ed95-4828-9828-c9715e22b66e service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:38.195767 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5baf8e2-ed95-4828-9828-c9715e22b66e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:38.197431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c5baf8e2-ed95-4828-9828-c9715e22b66e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:38.323283 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5baf8e2-ed95-4828-9828-c9715e22b66e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:38.324774 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5baf8e2-ed95-4828-9828-c9715e22b66e service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:38.344048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 727/1461] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:40.386046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612ab34-2275-4fe7-9d01-adee6247e517 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:40.386585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612ab34-2275-4fe7-9d01-adee6247e517 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:40.386585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612ab34-2275-4fe7-9d01-adee6247e517 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:40.388862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:17:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:40.395380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f38a39-123e-4e37-8ac9-84b57f510d86 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:40.395833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f38a39-123e-4e37-8ac9-84b57f510d86 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:40.395953 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f38a39-123e-4e37-8ac9-84b57f510d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:40.468761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e2f38a39-123e-4e37-8ac9-84b57f510d86 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:40.470022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e2f38a39-123e-4e37-8ac9-84b57f510d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:40.518258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:40.580917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:40.621286 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612ab34-2275-4fe7-9d01-adee6247e517 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:40.622885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e612ab34-2275-4fe7-9d01-adee6247e517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:40.745901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612ab34-2275-4fe7-9d01-adee6247e517 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:40.788504 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:40.827271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3714b4c-7937-4b31-8633-2c80aabf02c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:40.827904 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3714b4c-7937-4b31-8633-2c80aabf02c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:40.828179 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3714b4c-7937-4b31-8633-2c80aabf02c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:40.828441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3714b4c-7937-4b31-8633-2c80aabf02c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:40.830442 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-a3714b4c-7937-4b31-8633-2c80aabf02c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:40.947509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3714b4c-7937-4b31-8633-2c80aabf02c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:40.948945 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3714b4c-7937-4b31-8633-2c80aabf02c2 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:40.977095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 736/1464] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:43.008273 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d0692-dc81-49e0-a139-d281e3e9bb1b None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:43.009211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d0692-dc81-49e0-a139-d281e3e9bb1b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:43.009211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d0692-dc81-49e0-a139-d281e3e9bb1b None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:43.011256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:17:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:43.017313 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579b23a-1d49-4425-b57b-4f0a543aa752 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:43.017850 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579b23a-1d49-4425-b57b-4f0a543aa752 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:43.017971 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579b23a-1d49-4425-b57b-4f0a543aa752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:43.099014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-4579b23a-1d49-4425-b57b-4f0a543aa752 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:43.100500 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4579b23a-1d49-4425-b57b-4f0a543aa752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:43.121404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:43.203263 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:43.241411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d0692-dc81-49e0-a139-d281e3e9bb1b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:43.243779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f29d0692-dc81-49e0-a139-d281e3e9bb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:43.373860 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d0692-dc81-49e0-a139-d281e3e9bb1b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:43.448222 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:43.459336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494fc9a-bb17-4321-9a1c-6f717cb3295d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:43.460276 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494fc9a-bb17-4321-9a1c-6f717cb3295d service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:43.460674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494fc9a-bb17-4321-9a1c-6f717cb3295d service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:43.461028 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494fc9a-bb17-4321-9a1c-6f717cb3295d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:43.463728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2494fc9a-bb17-4321-9a1c-6f717cb3295d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:43.606116 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494fc9a-bb17-4321-9a1c-6f717cb3295d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:43.607582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494fc9a-bb17-4321-9a1c-6f717cb3295d service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:43.626754 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 730/1467] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:45.657077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861e68b-12df-4fdc-a5e3-5d419f7650be None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:45.657722 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861e68b-12df-4fdc-a5e3-5d419f7650be None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:45.657722 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861e68b-12df-4fdc-a5e3-5d419f7650be None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:45.659012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:17:45.665238 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fa22e-3c02-4b79-b323-2b186d973d35 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:45.665682 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fa22e-3c02-4b79-b323-2b186d973d35 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:45.665816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fa22e-3c02-4b79-b323-2b186d973d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:45.740076 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7e7fa22e-3c02-4b79-b323-2b186d973d35 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:45.741339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7e7fa22e-3c02-4b79-b323-2b186d973d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:45.797476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:45.849686 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:45.888359 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861e68b-12df-4fdc-a5e3-5d419f7650be None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:45.890826 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1861e68b-12df-4fdc-a5e3-5d419f7650be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:46.009258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861e68b-12df-4fdc-a5e3-5d419f7650be None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:46.059276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:46.088736 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2da33-c764-46fa-b8b2-05214b49b9af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:46.089369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2da33-c764-46fa-b8b2-05214b49b9af service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:46.089713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2da33-c764-46fa-b8b2-05214b49b9af service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:46.090010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2da33-c764-46fa-b8b2-05214b49b9af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:46.091654 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-06b2da33-c764-46fa-b8b2-05214b49b9af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:46.340569 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2da33-c764-46fa-b8b2-05214b49b9af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:46.342058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2da33-c764-46fa-b8b2-05214b49b9af service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:46.363981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 739/1470] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:17:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:55.847523 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=22 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:56.361279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=66 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:05.931469 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab6964-5cec-48dd-824b-d487a60448ef None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:05.932087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab6964-5cec-48dd-824b-d487a60448ef None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:05.932087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab6964-5cec-48dd-824b-d487a60448ef None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:05.933462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 732/1471] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:18:05.939458 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360a8f7-7e10-409e-ba27-c6add47d1fd6 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:05.939975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360a8f7-7e10-409e-ba27-c6add47d1fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:05.940088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360a8f7-7e10-409e-ba27-c6add47d1fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:06.011483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-3360a8f7-7e10-409e-ba27-c6add47d1fd6 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:06.012656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-3360a8f7-7e10-409e-ba27-c6add47d1fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:06.122428 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 740/1472] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:06.208230 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a180ae-ce96-4669-9434-838a02e6fdfe None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:06.208498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a180ae-ce96-4669-9434-838a02e6fdfe None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:06.208771 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a180ae-ce96-4669-9434-838a02e6fdfe None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:06.210184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 733/1473] 10.222.0.48 () {54 vars in 966 bytes} [Mon Aug 1 15:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:18:06.215666 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ef01e-9fea-42e8-b4a6-9dbdf6bb5564 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:06.215948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ef01e-9fea-42e8-b4a6-9dbdf6bb5564 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:06.216226 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ef01e-9fea-42e8-b4a6-9dbdf6bb5564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:06.295505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-fc7ef01e-9fea-42e8-b4a6-9dbdf6bb5564 None None] MFA Rules not processed for user `94f696dfe4b34c4aa19104b5f7ee6e70`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:06.296745 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-fc7ef01e-9fea-42e8-b4a6-9dbdf6bb5564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:06.318675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:06.381102 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 741/1474] 10.222.0.48 () {58 vars in 1063 bytes} [Mon Aug 1 15:18:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:18:06.389174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a180ae-ce96-4669-9434-838a02e6fdfe None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:06.390828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d4a180ae-ce96-4669-9434-838a02e6fdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:06.498120 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a180ae-ce96-4669-9434-838a02e6fdfe None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:06.573395 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effae2c5-7045-4122-8670-c088569385c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f696dfe4b34c4aa19104b5f7ee6e70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:06.573959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effae2c5-7045-4122-8670-c088569385c2 service glance] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:06.574236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effae2c5-7045-4122-8670-c088569385c2 service glance] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:06.574546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effae2c5-7045-4122-8670-c088569385c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:06.576194 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-effae2c5-7045-4122-8670-c088569385c2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:06.632865 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:06.690829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effae2c5-7045-4122-8670-c088569385c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:06.692335 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effae2c5-7045-4122-8670-c088569385c2 service glance] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:06.711641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 734/1475] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:18:08.773974 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2f095-4a3f-40a6-b9ae-c42621e68a34 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:08.774633 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2f095-4a3f-40a6-b9ae-c42621e68a34 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:08.774633 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2f095-4a3f-40a6-b9ae-c42621e68a34 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:08.775977 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 742/1476] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:18:08.781862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228e42c-d48a-4958-83f9-b2371beccc05 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:08.782298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228e42c-d48a-4958-83f9-b2371beccc05 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:08.782403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228e42c-d48a-4958-83f9-b2371beccc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:08.853866 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-0228e42c-d48a-4958-83f9-b2371beccc05 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:08.855082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0228e42c-d48a-4958-83f9-b2371beccc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:08.966572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 735/1477] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:18:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:08.993526 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2f095-4a3f-40a6-b9ae-c42621e68a34 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:08.995426 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8be2f095-4a3f-40a6-b9ae-c42621e68a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:09.153290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2f095-4a3f-40a6-b9ae-c42621e68a34 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:09.211264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:09.268309 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16104217-2d06-4f5e-af54-6efca0619d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:09.268944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16104217-2d06-4f5e-af54-6efca0619d2a admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:09.269219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16104217-2d06-4f5e-af54-6efca0619d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:09.269486 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16104217-2d06-4f5e-af54-6efca0619d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:09.270273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16104217-2d06-4f5e-af54-6efca0619d2a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:09.272057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: WARNING py.warnings [None req-16104217-2d06-4f5e-af54-6efca0619d2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:18:09.272057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: warnings.warn(msg) Aug 01 15:18:09.272057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: Aug 01 15:18:09.273196 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16104217-2d06-4f5e-af54-6efca0619d2a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:09.293710 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 743/1478] 10.222.0.48 () {56 vars in 1108 bytes} [Mon Aug 1 15:18:08 2022] GET /identity/v3/projects => generated 2553 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:18:11.456856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737429f-304b-4025-b56d-5e6cacfc39b3 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:11.457465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737429f-304b-4025-b56d-5e6cacfc39b3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:11.457465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737429f-304b-4025-b56d-5e6cacfc39b3 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:11.458977 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 736/1479] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:18:11.465528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2e9d9-826d-46ad-9705-b8effe4525a9 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:11.466366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2e9d9-826d-46ad-9705-b8effe4525a9 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:11.466525 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2e9d9-826d-46ad-9705-b8effe4525a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:11.553824 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-b2d2e9d9-826d-46ad-9705-b8effe4525a9 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:11.555108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-b2d2e9d9-826d-46ad-9705-b8effe4525a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:11.671036 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 744/1480] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:11.719283 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737429f-304b-4025-b56d-5e6cacfc39b3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:11.720827 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1737429f-304b-4025-b56d-5e6cacfc39b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:11.731049 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:11.884020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737429f-304b-4025-b56d-5e6cacfc39b3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:11.979233 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:12.016134 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacaad53-75eb-48b8-8168-ba750d7fa50f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:12.016763 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacaad53-75eb-48b8-8168-ba750d7fa50f service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:12.017043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacaad53-75eb-48b8-8168-ba750d7fa50f service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:12.017313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacaad53-75eb-48b8-8168-ba750d7fa50f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:12.019021 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-aacaad53-75eb-48b8-8168-ba750d7fa50f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:12.128034 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacaad53-75eb-48b8-8168-ba750d7fa50f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:12.129650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacaad53-75eb-48b8-8168-ba750d7fa50f service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:12.150124 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 737/1481] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:18:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:18:14.430497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8231a-4bf2-48d7-b866-ba32a25616af None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:14.431149 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8231a-4bf2-48d7-b866-ba32a25616af None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:14.431149 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8231a-4bf2-48d7-b866-ba32a25616af None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:14.432394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 745/1482] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:18:14.438514 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d43e7-e7ca-4d4a-b104-95fab41c22fb None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:14.439223 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d43e7-e7ca-4d4a-b104-95fab41c22fb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:14.439406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d43e7-e7ca-4d4a-b104-95fab41c22fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:14.513757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b24d43e7-e7ca-4d4a-b104-95fab41c22fb None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:14.515181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b24d43e7-e7ca-4d4a-b104-95fab41c22fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:14.564611 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:14.616542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 738/1483] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:14.653816 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8231a-4bf2-48d7-b866-ba32a25616af None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:14.656816 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-aae8231a-4bf2-48d7-b866-ba32a25616af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:14.693281 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:14.791803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae8231a-4bf2-48d7-b866-ba32a25616af None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:14.878261 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23577f6-1b74-4b95-86b0-8565a3c44971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:14.879054 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23577f6-1b74-4b95-86b0-8565a3c44971 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:14.879160 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23577f6-1b74-4b95-86b0-8565a3c44971 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:14.879417 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23577f6-1b74-4b95-86b0-8565a3c44971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:14.881057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-b23577f6-1b74-4b95-86b0-8565a3c44971 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:14.958954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:15.004661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23577f6-1b74-4b95-86b0-8565a3c44971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:15.006888 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23577f6-1b74-4b95-86b0-8565a3c44971 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:15.028659 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 746/1484] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:18:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:18:17.061712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcd444-7dfb-485e-9657-84a00d5a2ce9 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:17.062488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcd444-7dfb-485e-9657-84a00d5a2ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:17.062488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fcd444-7dfb-485e-9657-84a00d5a2ce9 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:17.063702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 739/1485] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:18:17.069606 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c239d6-b102-49d1-9bbd-93e0d71eeeae None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:17.070091 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c239d6-b102-49d1-9bbd-93e0d71eeeae None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:17.070189 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c239d6-b102-49d1-9bbd-93e0d71eeeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:17.149698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-69c239d6-b102-49d1-9bbd-93e0d71eeeae None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:17.150961 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-69c239d6-b102-49d1-9bbd-93e0d71eeeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:17.269521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 747/1486] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:17.316110 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fcd444-7dfb-485e-9657-84a00d5a2ce9 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:17.317725 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-06fcd444-7dfb-485e-9657-84a00d5a2ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:17.437195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fcd444-7dfb-485e-9657-84a00d5a2ce9 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:17.520863 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:17.523186 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c7c46-a5f3-4e5a-94de-e20b0b8cdb22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:17.523870 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c7c46-a5f3-4e5a-94de-e20b0b8cdb22 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:17.524151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c7c46-a5f3-4e5a-94de-e20b0b8cdb22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:17.524431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c7c46-a5f3-4e5a-94de-e20b0b8cdb22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:17.526924 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-009c7c46-a5f3-4e5a-94de-e20b0b8cdb22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:17.650698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c7c46-a5f3-4e5a-94de-e20b0b8cdb22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:17.652888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c7c46-a5f3-4e5a-94de-e20b0b8cdb22 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:17.682032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 740/1487] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:18:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:18:19.851442 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa59131-4575-474e-820f-6d4e95a76de9 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:19.852038 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa59131-4575-474e-820f-6d4e95a76de9 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:19.852038 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa59131-4575-474e-820f-6d4e95a76de9 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:19.853348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 748/1488] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:18:19.859915 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099c2e9-f57e-477b-8436-d8fd78efaa83 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:19.860692 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099c2e9-f57e-477b-8436-d8fd78efaa83 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:19.860839 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099c2e9-f57e-477b-8436-d8fd78efaa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:19.933745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6099c2e9-f57e-477b-8436-d8fd78efaa83 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:19.934990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6099c2e9-f57e-477b-8436-d8fd78efaa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:19.998539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:20.047285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 741/1489] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:18:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:20.091023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa59131-4575-474e-820f-6d4e95a76de9 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:20.092703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8fa59131-4575-474e-820f-6d4e95a76de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:20.131431 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:20.208276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa59131-4575-474e-820f-6d4e95a76de9 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:20.295917 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9be257-11f7-4441-aa54-2d1b28f5f00d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:20.296577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9be257-11f7-4441-aa54-2d1b28f5f00d service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:20.296862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9be257-11f7-4441-aa54-2d1b28f5f00d service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:20.297135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9be257-11f7-4441-aa54-2d1b28f5f00d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:20.298809 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-dc9be257-11f7-4441-aa54-2d1b28f5f00d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:20.381675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:20.415204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9be257-11f7-4441-aa54-2d1b28f5f00d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:20.417412 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9be257-11f7-4441-aa54-2d1b28f5f00d service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:20.437402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 749/1490] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:18:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:18:22.667122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f50ee8-146d-438f-93a8-596ddcb5dc3e None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:22.667825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f50ee8-146d-438f-93a8-596ddcb5dc3e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:22.667825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f50ee8-146d-438f-93a8-596ddcb5dc3e None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:22.669091 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 742/1491] 10.222.0.48 () {54 vars in 926 bytes} [Mon Aug 1 15:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:18:22.674885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69360fe-1f79-40d2-b65c-363c4b2307a5 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:22.675302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69360fe-1f79-40d2-b65c-363c4b2307a5 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:22.675417 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69360fe-1f79-40d2-b65c-363c4b2307a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:22.753008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-c69360fe-1f79-40d2-b65c-363c4b2307a5 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:22.754275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c69360fe-1f79-40d2-b65c-363c4b2307a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:22.887602 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 750/1492] 10.222.0.48 () {58 vars in 1023 bytes} [Mon Aug 1 15:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:22.931317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f50ee8-146d-438f-93a8-596ddcb5dc3e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:22.933724 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d9f50ee8-146d-438f-93a8-596ddcb5dc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:23.056753 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f50ee8-146d-438f-93a8-596ddcb5dc3e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:23.142880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b5aac-2ba4-48f9-8618-8784c4dd5b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:23.143508 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b5aac-2ba4-48f9-8618-8784c4dd5b22 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:23.143790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b5aac-2ba4-48f9-8618-8784c4dd5b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:23.144066 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b5aac-2ba4-48f9-8618-8784c4dd5b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:23.145747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-049b5aac-2ba4-48f9-8618-8784c4dd5b22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:23.155789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:23.274367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b5aac-2ba4-48f9-8618-8784c4dd5b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:23.275781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b5aac-2ba4-48f9-8618-8784c4dd5b22 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:23.295320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 743/1493] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:18:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:18:32.885228 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=82 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:33.292852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=50 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:50.804432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4eb1ee-7798-4bab-94b6-5ab20d32843c None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:50.805058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4eb1ee-7798-4bab-94b6-5ab20d32843c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:50.805058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4eb1ee-7798-4bab-94b6-5ab20d32843c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:50.887925 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2c4eb1ee-7798-4bab-94b6-5ab20d32843c None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:50.889227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2c4eb1ee-7798-4bab-94b6-5ab20d32843c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:50.995733 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 751/1494] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:51.004453 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ba7d6-9e59-48b4-a763-1d13635c443e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.004992 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ba7d6-9e59-48b4-a763-1d13635c443e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.004992 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ba7d6-9e59-48b4-a763-1d13635c443e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.084508 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-569ba7d6-9e59-48b4-a763-1d13635c443e None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:51.085687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-569ba7d6-9e59-48b4-a763-1d13635c443e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:51.208360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 744/1495] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:51.215723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4eb1ee-7798-4bab-94b6-5ab20d32843c None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.216981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2c4eb1ee-7798-4bab-94b6-5ab20d32843c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:51.232826 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4eb1ee-7798-4bab-94b6-5ab20d32843c None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.299860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:51.343451 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73504d-e6de-41c7-89c2-e3c0fa789d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.344018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73504d-e6de-41c7-89c2-e3c0fa789d5d admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.344298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73504d-e6de-41c7-89c2-e3c0fa789d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.344556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73504d-e6de-41c7-89c2-e3c0fa789d5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.345619 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73504d-e6de-41c7-89c2-e3c0fa789d5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:51.345935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73504d-e6de-41c7-89c2-e3c0fa789d5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:51.347016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: WARNING py.warnings [None req-2a73504d-e6de-41c7-89c2-e3c0fa789d5d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:19:51.347016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: warnings.warn(msg) Aug 01 15:19:51.347016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: Aug 01 15:19:51.347932 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73504d-e6de-41c7-89c2-e3c0fa789d5d admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:51.362047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 752/1496] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:51.370499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ba7d6-9e59-48b4-a763-1d13635c443e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.380682 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ba7d6-9e59-48b4-a763-1d13635c443e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.451922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:51.492584 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3b7a8-75bf-43b1-a285-40ec3bdc1ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.493569 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3b7a8-75bf-43b1-a285-40ec3bdc1ffe admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.494019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3b7a8-75bf-43b1-a285-40ec3bdc1ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.494426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3b7a8-75bf-43b1-a285-40ec3bdc1ffe admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.495493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3b7a8-75bf-43b1-a285-40ec3bdc1ffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:51.497899 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: WARNING py.warnings [None req-1dd3b7a8-75bf-43b1-a285-40ec3bdc1ffe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:19:51.497899 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: warnings.warn(msg) Aug 01 15:19:51.497899 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: Aug 01 15:19:51.499280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3b7a8-75bf-43b1-a285-40ec3bdc1ffe admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:51.576073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 745/1497] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:19:51 2022] POST /identity/v3/projects => generated 379 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:51.580427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73504d-e6de-41c7-89c2-e3c0fa789d5d admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.581992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2a73504d-e6de-41c7-89c2-e3c0fa789d5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:51.620519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73504d-e6de-41c7-89c2-e3c0fa789d5d admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.735251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bb049-e337-4ab4-be25-05202cd59d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.735958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bb049-e337-4ab4-be25-05202cd59d38 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.737904 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bb049-e337-4ab4-be25-05202cd59d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.737904 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bb049-e337-4ab4-be25-05202cd59d38 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.737904 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96bb049-e337-4ab4-be25-05202cd59d38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:51.738187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: WARNING py.warnings [None req-f96bb049-e337-4ab4-be25-05202cd59d38 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:19:51.738187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: warnings.warn(msg) Aug 01 15:19:51.738187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: Aug 01 15:19:51.739378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96bb049-e337-4ab4-be25-05202cd59d38 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:51.784058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 753/1498] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:19:51 2022] POST /identity/v3/users => generated 290 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:51.790376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3b7a8-75bf-43b1-a285-40ec3bdc1ffe admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.792017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1dd3b7a8-75bf-43b1-a285-40ec3bdc1ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:51.866280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3b7a8-75bf-43b1-a285-40ec3bdc1ffe admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:51.956405 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:51.969476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf726be5-dfd0-49d1-9573-123441415b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:51.970503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf726be5-dfd0-49d1-9573-123441415b19 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:51.970779 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf726be5-dfd0-49d1-9573-123441415b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:51.971046 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf726be5-dfd0-49d1-9573-123441415b19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:51.971768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf726be5-dfd0-49d1-9573-123441415b19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:51.972809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: WARNING py.warnings [None req-cf726be5-dfd0-49d1-9573-123441415b19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:19:51.972809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: warnings.warn(msg) Aug 01 15:19:51.972809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: Aug 01 15:19:51.973861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf726be5-dfd0-49d1-9573-123441415b19 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:51.987659 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 746/1499] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:51.993631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bb049-e337-4ab4-be25-05202cd59d38 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:51.993631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f96bb049-e337-4ab4-be25-05202cd59d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:52.005190 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bb049-e337-4ab4-be25-05202cd59d38 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:52.032893 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:52.142717 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720c803-d2c6-4d30-8241-653265d5b6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:52.143276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720c803-d2c6-4d30-8241-653265d5b6eb admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.143556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720c803-d2c6-4d30-8241-653265d5b6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.143827 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720c803-d2c6-4d30-8241-653265d5b6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.144524 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720c803-d2c6-4d30-8241-653265d5b6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:52.145570 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: WARNING py.warnings [None req-b720c803-d2c6-4d30-8241-653265d5b6eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:19:52.145570 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: warnings.warn(msg) Aug 01 15:19:52.145570 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: Aug 01 15:19:52.146498 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720c803-d2c6-4d30-8241-653265d5b6eb admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:52.160325 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 754/1500] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:52.165422 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf726be5-dfd0-49d1-9573-123441415b19 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:52.167098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cf726be5-dfd0-49d1-9573-123441415b19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:52.204265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf726be5-dfd0-49d1-9573-123441415b19 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:52.315212 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334115c5-1901-4a92-ad4b-ebb33735b5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:52.315907 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334115c5-1901-4a92-ad4b-ebb33735b5d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.316181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334115c5-1901-4a92-ad4b-ebb33735b5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.316455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334115c5-1901-4a92-ad4b-ebb33735b5d6 admin admin] PATH_INFO: `/v3/projects/4895f1ba5e28452e87b9c192fb20790b/users/8239b01f7f824b4db56bf70449257ad8/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.343218 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334115c5-1901-4a92-ad4b-ebb33735b5d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4895f1ba5e28452e87b9c192fb20790b, user_id=8239b01f7f824b4db56bf70449257ad8, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:52.344311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: WARNING py.warnings [None req-334115c5-1901-4a92-ad4b-ebb33735b5d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:19:52.344311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: warnings.warn(msg) Aug 01 15:19:52.344311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: Aug 01 15:19:52.346714 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334115c5-1901-4a92-ad4b-ebb33735b5d6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:52.377036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 747/1501] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:19:52 2022] PUT /identity/v3/projects/4895f1ba5e28452e87b9c192fb20790b/users/8239b01f7f824b4db56bf70449257ad8/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:52.390543 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63286ef-0e53-4e35-af9c-ed539308e14e None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.390854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63286ef-0e53-4e35-af9c-ed539308e14e None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.391239 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63286ef-0e53-4e35-af9c-ed539308e14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.451147 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-a63286ef-0e53-4e35-af9c-ed539308e14e None None] MFA Rules not processed for user `8239b01f7f824b4db56bf70449257ad8`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:52.452292 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-a63286ef-0e53-4e35-af9c-ed539308e14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:52.541164 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 755/1502] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:52.556082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b90e9e-b3f4-41a1-afc3-70e51e7a352e None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.556449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b90e9e-b3f4-41a1-afc3-70e51e7a352e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.556859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b90e9e-b3f4-41a1-afc3-70e51e7a352e None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.559005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 748/1503] 10.222.0.48 () {54 vars in 964 bytes} [Mon Aug 1 15:19:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:52.563946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83982373-df97-4dfa-a000-6222ba330d79 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:52.564229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83982373-df97-4dfa-a000-6222ba330d79 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:52.564531 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83982373-df97-4dfa-a000-6222ba330d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:52.599960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:52.651345 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-83982373-df97-4dfa-a000-6222ba330d79 None None] MFA Rules not processed for user `63824ba9369c4899a4b946588cbdae24`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:52.652590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-83982373-df97-4dfa-a000-6222ba330d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:52.738740 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 756/1504] 10.222.0.48 () {58 vars in 1061 bytes} [Mon Aug 1 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:52.749334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b90e9e-b3f4-41a1-afc3-70e51e7a352e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:52.750954 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-44b90e9e-b3f4-41a1-afc3-70e51e7a352e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:52.806165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:52.864728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b90e9e-b3f4-41a1-afc3-70e51e7a352e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:53.079798 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322bf87-d322-401e-aa69-d5f30336f3b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:53.080507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322bf87-d322-401e-aa69-d5f30336f3b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.080997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322bf87-d322-401e-aa69-d5f30336f3b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.081343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322bf87-d322-401e-aa69-d5f30336f3b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.083036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9322bf87-d322-401e-aa69-d5f30336f3b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:53.191305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322bf87-d322-401e-aa69-d5f30336f3b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:53.192815 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322bf87-d322-401e-aa69-d5f30336f3b1 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:53.196645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:53.213847 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 749/1505] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:53.260467 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8effc86-ab46-4fc6-99ce-6bc9cb80c652 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.260916 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8effc86-ab46-4fc6-99ce-6bc9cb80c652 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.261040 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8effc86-ab46-4fc6-99ce-6bc9cb80c652 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.262337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 757/1506] 10.222.0.48 () {52 vars in 830 bytes} [Mon Aug 1 15:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:19:53.266332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322bf87-d322-401e-aa69-d5f30336f3b1 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:53.267569 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9322bf87-d322-401e-aa69-d5f30336f3b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:53.376036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322bf87-d322-401e-aa69-d5f30336f3b1 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:53.482551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b0edc-6178-4ce1-b4e8-9cbab41eac8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:53.483201 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b0edc-6178-4ce1-b4e8-9cbab41eac8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.483467 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b0edc-6178-4ce1-b4e8-9cbab41eac8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.483754 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b0edc-6178-4ce1-b4e8-9cbab41eac8c admin admin] PATH_INFO: `/v3/users/8239b01f7f824b4db56bf70449257ad8` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.498382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b0edc-6178-4ce1-b4e8-9cbab41eac8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8239b01f7f824b4db56bf70449257ad8)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:53.499412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: WARNING py.warnings [None req-eb8b0edc-6178-4ce1-b4e8-9cbab41eac8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:19:53.499412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: warnings.warn(msg) Aug 01 15:19:53.499412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: Aug 01 15:19:53.500633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b0edc-6178-4ce1-b4e8-9cbab41eac8c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:53.511035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:53.575163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-eb8b0edc-6178-4ce1-b4e8-9cbab41eac8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8239b01f7f824b4db56bf70449257ad8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:53.575705 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-eb8b0edc-6178-4ce1-b4e8-9cbab41eac8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8239b01f7f824b4db56bf70449257ad8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:53.602364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-eb8b0edc-6178-4ce1-b4e8-9cbab41eac8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8239b01f7f824b4db56bf70449257ad8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:53.614169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-eb8b0edc-6178-4ce1-b4e8-9cbab41eac8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8239b01f7f824b4db56bf70449257ad8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:53.636547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-eb8b0edc-6178-4ce1-b4e8-9cbab41eac8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8239b01f7f824b4db56bf70449257ad8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:53.643404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 750/1507] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:19:53 2022] DELETE /identity/v3/users/8239b01f7f824b4db56bf70449257ad8 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:53.648456 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706d05e-8933-44ed-a62c-0d99ba275ed7 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:53.648740 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706d05e-8933-44ed-a62c-0d99ba275ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:53.649117 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706d05e-8933-44ed-a62c-0d99ba275ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:53.721664 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-e706d05e-8933-44ed-a62c-0d99ba275ed7 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:53.722928 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e706d05e-8933-44ed-a62c-0d99ba275ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:53.804195 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:53.821848 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 758/1508] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:53.832106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b0edc-6178-4ce1-b4e8-9cbab41eac8c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:53.833343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-eb8b0edc-6178-4ce1-b4e8-9cbab41eac8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:53.952177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b0edc-6178-4ce1-b4e8-9cbab41eac8c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:54.016374 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:54.040331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafb80f-091c-4ddd-af98-f14443ba83da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:54.041053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafb80f-091c-4ddd-af98-f14443ba83da service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.041333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafb80f-091c-4ddd-af98-f14443ba83da service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.041598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafb80f-091c-4ddd-af98-f14443ba83da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.043299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6dafb80f-091c-4ddd-af98-f14443ba83da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:54.160320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dafb80f-091c-4ddd-af98-f14443ba83da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:54.161685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dafb80f-091c-4ddd-af98-f14443ba83da service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:54.181015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 751/1509] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:19:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:54.541423 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706d05e-8933-44ed-a62c-0d99ba275ed7 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:54.542717 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e706d05e-8933-44ed-a62c-0d99ba275ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:54.554293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706d05e-8933-44ed-a62c-0d99ba275ed7 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:54.662847 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ade338-ba7e-45e7-9028-94a03a53b1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:54.663508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ade338-ba7e-45e7-9028-94a03a53b1e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:54.663782 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ade338-ba7e-45e7-9028-94a03a53b1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:54.664055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ade338-ba7e-45e7-9028-94a03a53b1e1 admin admin] PATH_INFO: `/v3/projects/4895f1ba5e28452e87b9c192fb20790b` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:54.675249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ade338-ba7e-45e7-9028-94a03a53b1e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4895f1ba5e28452e87b9c192fb20790b)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:54.676331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: WARNING py.warnings [None req-56ade338-ba7e-45e7-9028-94a03a53b1e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:19:54.676331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: warnings.warn(msg) Aug 01 15:19:54.676331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: Aug 01 15:19:54.677537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ade338-ba7e-45e7-9028-94a03a53b1e1 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:54.752715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-56ade338-ba7e-45e7-9028-94a03a53b1e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4895f1ba5e28452e87b9c192fb20790b', 'request_id': 'req-56ade338-ba7e-45e7-9028-94a03a53b1e1'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:54.765427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-56ade338-ba7e-45e7-9028-94a03a53b1e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4895f1ba5e28452e87b9c192fb20790b', 'request_id': 'req-56ade338-ba7e-45e7-9028-94a03a53b1e1'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:54.783503 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-56ade338-ba7e-45e7-9028-94a03a53b1e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4895f1ba5e28452e87b9c192fb20790b', 'request_id': 'req-56ade338-ba7e-45e7-9028-94a03a53b1e1'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:54.809251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-56ade338-ba7e-45e7-9028-94a03a53b1e1 admin admin] The token cache is being invalidate because project 4895f1ba5e28452e87b9c192fb20790b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:54.809412 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-56ade338-ba7e-45e7-9028-94a03a53b1e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:54.809877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-56ade338-ba7e-45e7-9028-94a03a53b1e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:54.811351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 759/1510] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:19:54 2022] DELETE /identity/v3/projects/4895f1ba5e28452e87b9c192fb20790b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:04.178579 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=62 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:04.807157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:41.088044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99523aa8-489f-4af8-9e3f-4e8debdf54ab None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:41.088676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99523aa8-489f-4af8-9e3f-4e8debdf54ab None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:41.088676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99523aa8-489f-4af8-9e3f-4e8debdf54ab None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:41.090144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:41.095687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818088bf-83b5-4ee9-a39e-5ca69ca779da None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:41.096143 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818088bf-83b5-4ee9-a39e-5ca69ca779da None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:41.096247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818088bf-83b5-4ee9-a39e-5ca69ca779da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:41.193028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-818088bf-83b5-4ee9-a39e-5ca69ca779da None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:41.194228 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-818088bf-83b5-4ee9-a39e-5ca69ca779da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:41.325566 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:41.484205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99523aa8-489f-4af8-9e3f-4e8debdf54ab None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:41.487120 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-99523aa8-489f-4af8-9e3f-4e8debdf54ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:41.621088 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99523aa8-489f-4af8-9e3f-4e8debdf54ab None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:41.627224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079091a-ba96-4e04-b159-2e5a0cbe0262 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:41.627896 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079091a-ba96-4e04-b159-2e5a0cbe0262 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:41.628239 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079091a-ba96-4e04-b159-2e5a0cbe0262 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:41.628511 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079091a-ba96-4e04-b159-2e5a0cbe0262 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:41.637431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079091a-ba96-4e04-b159-2e5a0cbe0262 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:41.639218 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079091a-ba96-4e04-b159-2e5a0cbe0262 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:41.649501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 753/1513] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:21:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:21:42.489069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff94f0-3649-4a1e-9a0b-76d50bdf8392 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:42.489889 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff94f0-3649-4a1e-9a0b-76d50bdf8392 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:42.490019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff94f0-3649-4a1e-9a0b-76d50bdf8392 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:42.491803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:21:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:42.497232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac5a26-66ff-485c-abdb-170c2b6c0047 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:42.497898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac5a26-66ff-485c-abdb-170c2b6c0047 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:42.498189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac5a26-66ff-485c-abdb-170c2b6c0047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:42.521261 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3dac5a26-66ff-485c-abdb-170c2b6c0047 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:42.522616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3dac5a26-66ff-485c-abdb-170c2b6c0047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:42.531589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:42.691343 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff94f0-3649-4a1e-9a0b-76d50bdf8392 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:42.708084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff94f0-3649-4a1e-9a0b-76d50bdf8392 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:42.715351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8725400-5486-43d4-90f3-7aa57adfcaaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:42.715981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8725400-5486-43d4-90f3-7aa57adfcaaf service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:42.716258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8725400-5486-43d4-90f3-7aa57adfcaaf service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:42.716533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8725400-5486-43d4-90f3-7aa57adfcaaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:42.726234 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8725400-5486-43d4-90f3-7aa57adfcaaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:42.727894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8725400-5486-43d4-90f3-7aa57adfcaaf service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:42.738194 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 762/1516] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:21:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:21:44.251412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771a83b-f8d2-4871-bec8-3413f01826dd None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:44.252256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771a83b-f8d2-4871-bec8-3413f01826dd None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:44.252256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771a83b-f8d2-4871-bec8-3413f01826dd None None] PATH_INFO: `/v3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:44.253217 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 755/1517] 10.222.0.58 () {54 vars in 925 bytes} [Mon Aug 1 15:21:44 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:21:44.261314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fc695-353d-42d3-8ded-335c5cc6d5ce None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:44.261845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fc695-353d-42d3-8ded-335c5cc6d5ce None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:44.261845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fc695-353d-42d3-8ded-335c5cc6d5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:44.327319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-9d6fc695-353d-42d3-8ded-335c5cc6d5ce None None] MFA Rules not processed for user `098d8d7e35804ccc931ddfba07f688a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:44.328566 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9d6fc695-353d-42d3-8ded-335c5cc6d5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:44.358138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:44.427659 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 763/1518] 10.222.0.58 () {58 vars in 1015 bytes} [Mon Aug 1 15:21:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:48.505953 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56447ba0-ca74-4186-974f-aa0ebf0451e0 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:48.506812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56447ba0-ca74-4186-974f-aa0ebf0451e0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:48.506812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56447ba0-ca74-4186-974f-aa0ebf0451e0 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:48.508535 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 756/1519] 10.222.0.48 () {54 vars in 961 bytes} [Mon Aug 1 15:21:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:48.516918 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbac4d-c0f5-4635-91ec-d726b2ef3c82 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:48.517364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbac4d-c0f5-4635-91ec-d726b2ef3c82 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:48.517565 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbac4d-c0f5-4635-91ec-d726b2ef3c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:48.538589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-5ccbac4d-c0f5-4635-91ec-d726b2ef3c82 None None] MFA Rules not processed for user `098d8d7e35804ccc931ddfba07f688a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:48.539803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5ccbac4d-c0f5-4635-91ec-d726b2ef3c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:48.548815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 764/1520] 10.222.0.48 () {58 vars in 1058 bytes} [Mon Aug 1 15:21:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:48.557140 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56447ba0-ca74-4186-974f-aa0ebf0451e0 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:48.578971 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56447ba0-ca74-4186-974f-aa0ebf0451e0 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:48.586548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec6f15-a903-4fb8-b24d-3237bc42454c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '098d8d7e35804ccc931ddfba07f688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:48.587270 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec6f15-a903-4fb8-b24d-3237bc42454c service placement] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:48.587563 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec6f15-a903-4fb8-b24d-3237bc42454c service placement] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:48.587924 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec6f15-a903-4fb8-b24d-3237bc42454c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:48.597380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec6f15-a903-4fb8-b24d-3237bc42454c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:48.599103 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec6f15-a903-4fb8-b24d-3237bc42454c service placement] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:48.609192 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 757/1521] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:21:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:21:54.059530 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c36e3-ec3c-4d89-a496-8f177a0b54eb None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:54.060159 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c36e3-ec3c-4d89-a496-8f177a0b54eb None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:54.060159 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c36e3-ec3c-4d89-a496-8f177a0b54eb None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:54.061954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:21:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:54.069726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f793c82-713b-404e-9e61-8d3a8f0c1380 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:54.070504 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f793c82-713b-404e-9e61-8d3a8f0c1380 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:54.070820 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f793c82-713b-404e-9e61-8d3a8f0c1380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:54.103301 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4f793c82-713b-404e-9e61-8d3a8f0c1380 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:54.104777 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4f793c82-713b-404e-9e61-8d3a8f0c1380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:54.116744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:54.277494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c36e3-ec3c-4d89-a496-8f177a0b54eb None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:54.297118 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c36e3-ec3c-4d89-a496-8f177a0b54eb None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:54.303292 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe9c2f-4ff2-4e2b-93ee-357685ced4de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:54.303922 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe9c2f-4ff2-4e2b-93ee-357685ced4de service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:54.304361 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe9c2f-4ff2-4e2b-93ee-357685ced4de service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:54.304641 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe9c2f-4ff2-4e2b-93ee-357685ced4de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:54.314121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe9c2f-4ff2-4e2b-93ee-357685ced4de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:54.315733 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe9c2f-4ff2-4e2b-93ee-357685ced4de service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:54.329861 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 766/1524] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:21:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:21:55.185657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb942a-cf3e-47b4-806f-c4c7963cc74d None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:55.186287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb942a-cf3e-47b4-806f-c4c7963cc74d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:55.186287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb942a-cf3e-47b4-806f-c4c7963cc74d None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:55.187651 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:55.198260 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6f1fa-6764-44b1-b64c-3808458d7fbf None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:55.199014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6f1fa-6764-44b1-b64c-3808458d7fbf None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:55.199150 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6f1fa-6764-44b1-b64c-3808458d7fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:55.224953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-19a6f1fa-6764-44b1-b64c-3808458d7fbf None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:55.226221 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-19a6f1fa-6764-44b1-b64c-3808458d7fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:55.235596 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:55.397201 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb942a-cf3e-47b4-806f-c4c7963cc74d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:55.412757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb942a-cf3e-47b4-806f-c4c7963cc74d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:55.419058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e2720-e4ef-411a-8647-b0385971de14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:55.419680 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e2720-e4ef-411a-8647-b0385971de14 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:55.419955 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e2720-e4ef-411a-8647-b0385971de14 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:55.420224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e2720-e4ef-411a-8647-b0385971de14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:55.429292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e2720-e4ef-411a-8647-b0385971de14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:55.430925 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e2720-e4ef-411a-8647-b0385971de14 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:55.441254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 760/1527] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:21:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:21:59.385514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3803c1-9570-4fab-bed5-00565c8675f2 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:59.386094 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3803c1-9570-4fab-bed5-00565c8675f2 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:59.386094 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3803c1-9570-4fab-bed5-00565c8675f2 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:59.387473 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:21:59.393387 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a69786-25dc-49bd-bd1b-2f9db5114449 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:59.394154 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a69786-25dc-49bd-bd1b-2f9db5114449 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:59.394154 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a69786-25dc-49bd-bd1b-2f9db5114449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:59.415948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-48a69786-25dc-49bd-bd1b-2f9db5114449 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:59.417158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-48a69786-25dc-49bd-bd1b-2f9db5114449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:59.425944 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:59.503314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3803c1-9570-4fab-bed5-00565c8675f2 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:59.505187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-cb3803c1-9570-4fab-bed5-00565c8675f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:59.591424 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3803c1-9570-4fab-bed5-00565c8675f2 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:59.595848 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6d6f5-7f85-4848-bc4e-cdf0da0fe29a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:59.596549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6d6f5-7f85-4848-bc4e-cdf0da0fe29a service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:59.596766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6d6f5-7f85-4848-bc4e-cdf0da0fe29a service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:59.597041 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6d6f5-7f85-4848-bc4e-cdf0da0fe29a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:59.607107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6d6f5-7f85-4848-bc4e-cdf0da0fe29a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:59.609016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6d6f5-7f85-4848-bc4e-cdf0da0fe29a service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:59.618985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 769/1530] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:21:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:22:01.629512 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9056a6-b9a6-45e9-8025-215d068e50c3 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:01.630121 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9056a6-b9a6-45e9-8025-215d068e50c3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:01.630121 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9056a6-b9a6-45e9-8025-215d068e50c3 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:01.631677 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:22:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:22:01.637433 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32728cbd-8e0b-4114-a68f-fce449c9951f None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:01.638106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32728cbd-8e0b-4114-a68f-fce449c9951f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:01.638106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32728cbd-8e0b-4114-a68f-fce449c9951f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:01.660133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-32728cbd-8e0b-4114-a68f-fce449c9951f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:01.661388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-32728cbd-8e0b-4114-a68f-fce449c9951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:01.670532 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:22:01.718755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9056a6-b9a6-45e9-8025-215d068e50c3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:01.721035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7a9056a6-b9a6-45e9-8025-215d068e50c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:01.744056 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9056a6-b9a6-45e9-8025-215d068e50c3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:01.748668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b49d2-e145-4d65-bade-7011eb358aa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:01.751158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b49d2-e145-4d65-bade-7011eb358aa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:01.751434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b49d2-e145-4d65-bade-7011eb358aa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:01.751701 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b49d2-e145-4d65-bade-7011eb358aa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:01.761664 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b49d2-e145-4d65-bade-7011eb358aa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:01.763152 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b49d2-e145-4d65-bade-7011eb358aa6 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:01.773148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 763/1533] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:22:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:22:05.418448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17941e-c772-4467-bba9-30a4da440ad8 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:05.419049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17941e-c772-4467-bba9-30a4da440ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:05.419049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17941e-c772-4467-bba9-30a4da440ad8 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:05.420379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:22:05.426011 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526d159-e1d5-457e-9708-32b7376d365e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:05.426512 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526d159-e1d5-457e-9708-32b7376d365e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:05.426630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526d159-e1d5-457e-9708-32b7376d365e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:05.435244 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=81 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:05.450235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4526d159-e1d5-457e-9708-32b7376d365e None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:05.451498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4526d159-e1d5-457e-9708-32b7376d365e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:05.460180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:22:05.777519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17941e-c772-4467-bba9-30a4da440ad8 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:05.779346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2f17941e-c772-4467-bba9-30a4da440ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:05.857473 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:05.877706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17941e-c772-4467-bba9-30a4da440ad8 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:05.883321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea47085-7594-4cb2-b312-11c9b431a5a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:05.884217 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea47085-7594-4cb2-b312-11c9b431a5a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:05.884651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea47085-7594-4cb2-b312-11c9b431a5a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:05.885061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea47085-7594-4cb2-b312-11c9b431a5a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:05.895906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea47085-7594-4cb2-b312-11c9b431a5a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:05.897373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea47085-7594-4cb2-b312-11c9b431a5a0 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:05.908165 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 772/1536] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:22:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:22:07.955474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf71ff-92fa-4c8c-aadb-c67517b017c3 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:07.956061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf71ff-92fa-4c8c-aadb-c67517b017c3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:07.956061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf71ff-92fa-4c8c-aadb-c67517b017c3 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:07.957412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:22:07.962940 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10125ea0-e01c-44c4-a988-d15bc48c1b50 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:07.963429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10125ea0-e01c-44c4-a988-d15bc48c1b50 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:07.963527 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10125ea0-e01c-44c4-a988-d15bc48c1b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:07.986085 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-10125ea0-e01c-44c4-a988-d15bc48c1b50 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:07.987323 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-10125ea0-e01c-44c4-a988-d15bc48c1b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:07.996219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:22:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:22:08.303186 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf71ff-92fa-4c8c-aadb-c67517b017c3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:08.305107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6dcf71ff-92fa-4c8c-aadb-c67517b017c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:08.319343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf71ff-92fa-4c8c-aadb-c67517b017c3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:08.323396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6697de-7169-4d87-b653-e46825063a2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:08.324035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6697de-7169-4d87-b653-e46825063a2b service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:08.324305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6697de-7169-4d87-b653-e46825063a2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:08.324581 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6697de-7169-4d87-b653-e46825063a2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:08.335431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6697de-7169-4d87-b653-e46825063a2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:08.336965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6697de-7169-4d87-b653-e46825063a2b service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:08.350778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 766/1539] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:22:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:22:17.980596 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=11 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:18.343558 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=8 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:38.908626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029ee2e-557a-4493-859d-27d91256a5cd None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:38.909753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029ee2e-557a-4493-859d-27d91256a5cd None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:38.910272 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029ee2e-557a-4493-859d-27d91256a5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:38.937618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2029ee2e-557a-4493-859d-27d91256a5cd None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:38.939294 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2029ee2e-557a-4493-859d-27d91256a5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:38.952693 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 774/1540] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:38.962091 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638d97f-1acc-4d63-a318-ad20cc8ec483 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:38.963125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638d97f-1acc-4d63-a318-ad20cc8ec483 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:38.963614 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638d97f-1acc-4d63-a318-ad20cc8ec483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:38.991732 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a638d97f-1acc-4d63-a318-ad20cc8ec483 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:38.993599 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a638d97f-1acc-4d63-a318-ad20cc8ec483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:39.008802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 767/1541] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:39.014126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029ee2e-557a-4493-859d-27d91256a5cd None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.026070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029ee2e-557a-4493-859d-27d91256a5cd None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:39.036583 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231d4f9-5d4d-444e-8b67-420eb91aa1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:39.037629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231d4f9-5d4d-444e-8b67-420eb91aa1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.038500 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231d4f9-5d4d-444e-8b67-420eb91aa1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.038995 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231d4f9-5d4d-444e-8b67-420eb91aa1d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.039942 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231d4f9-5d4d-444e-8b67-420eb91aa1d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:39.040465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231d4f9-5d4d-444e-8b67-420eb91aa1d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:39.042382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231d4f9-5d4d-444e-8b67-420eb91aa1d8 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:39.057205 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 775/1542] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:39.066499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638d97f-1acc-4d63-a318-ad20cc8ec483 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.075242 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2ae27-b387-4bab-94a2-044b361a0e95 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.076039 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2ae27-b387-4bab-94a2-044b361a0e95 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.078461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638d97f-1acc-4d63-a318-ad20cc8ec483 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:39.082084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2ae27-b387-4bab-94a2-044b361a0e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.091400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0533a19-707f-4c27-8d26-91c13b250432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:39.092390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0533a19-707f-4c27-8d26-91c13b250432 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.093739 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0533a19-707f-4c27-8d26-91c13b250432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.094263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0533a19-707f-4c27-8d26-91c13b250432 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.096240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0533a19-707f-4c27-8d26-91c13b250432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:39.103508 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0533a19-707f-4c27-8d26-91c13b250432 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:39.127786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-10b2ae27-b387-4bab-94a2-044b361a0e95 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:39.131392 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-10b2ae27-b387-4bab-94a2-044b361a0e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:39.153496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 776/1543] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:39.156809 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c805b61-29c1-4133-83d1-c257efe9fb8b None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.157312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c805b61-29c1-4133-83d1-c257efe9fb8b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.157763 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c805b61-29c1-4133-83d1-c257efe9fb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.191500 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-6c805b61-29c1-4133-83d1-c257efe9fb8b None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:39.197805 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6c805b61-29c1-4133-83d1-c257efe9fb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:39.198046 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 768/1544] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:39.198046 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04ecd9-8b72-4c9b-bc42-ddb2b469366f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.198632 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04ecd9-8b72-4c9b-bc42-ddb2b469366f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.199016 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04ecd9-8b72-4c9b-bc42-ddb2b469366f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.233082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-cf04ecd9-8b72-4c9b-bc42-ddb2b469366f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:39.234312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cf04ecd9-8b72-4c9b-bc42-ddb2b469366f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:39.284869 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 777/1545] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:39.285730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c805b61-29c1-4133-83d1-c257efe9fb8b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.307390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c805b61-29c1-4133-83d1-c257efe9fb8b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:39.315491 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febe2af-c4c6-4e92-b118-f23abf8f377d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:39.316126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febe2af-c4c6-4e92-b118-f23abf8f377d admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.316399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febe2af-c4c6-4e92-b118-f23abf8f377d admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.316671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febe2af-c4c6-4e92-b118-f23abf8f377d admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.317345 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4febe2af-c4c6-4e92-b118-f23abf8f377d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:39.319540 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4febe2af-c4c6-4e92-b118-f23abf8f377d admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:39.325325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 769/1546] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:39.327631 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cdd9c-f57d-4ed0-8b43-ac0ad2bb8450 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.327631 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cdd9c-f57d-4ed0-8b43-ac0ad2bb8450 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.327631 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0cdd9c-f57d-4ed0-8b43-ac0ad2bb8450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.359340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-fe0cdd9c-f57d-4ed0-8b43-ac0ad2bb8450 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:39.359966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 778/1547] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/users => generated 283 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:39.360540 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-fe0cdd9c-f57d-4ed0-8b43-ac0ad2bb8450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:39.360902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d15f46-9c29-4d2b-9ff7-c25bf5cb74e6 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.361191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d15f46-9c29-4d2b-9ff7-c25bf5cb74e6 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.361465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d15f46-9c29-4d2b-9ff7-c25bf5cb74e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.373121 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 770/1548] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:39.377011 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e1cf0-ca1a-40bd-a40f-f35e12a103ca None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.377011 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e1cf0-ca1a-40bd-a40f-f35e12a103ca None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.377011 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e1cf0-ca1a-40bd-a40f-f35e12a103ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.384700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-20d15f46-9c29-4d2b-9ff7-c25bf5cb74e6 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:39.386044 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-20d15f46-9c29-4d2b-9ff7-c25bf5cb74e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:39.395019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 779/1549] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:39.395175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-694e1cf0-ca1a-40bd-a40f-f35e12a103ca None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:39.396192 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d15f46-9c29-4d2b-9ff7-c25bf5cb74e6 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.396331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-694e1cf0-ca1a-40bd-a40f-f35e12a103ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:39.406131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 771/1550] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:39.406361 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d15f46-9c29-4d2b-9ff7-c25bf5cb74e6 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:39.407420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d69ae-d83e-4dd0-80df-cc073d767f5c None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.407698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d69ae-d83e-4dd0-80df-cc073d767f5c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.407976 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d69ae-d83e-4dd0-80df-cc073d767f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.415182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e50bc-637a-4dec-85d9-008234ff92aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:39.415828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e50bc-637a-4dec-85d9-008234ff92aa admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.416138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e50bc-637a-4dec-85d9-008234ff92aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.417107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e50bc-637a-4dec-85d9-008234ff92aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.417985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3e50bc-637a-4dec-85d9-008234ff92aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:39.418316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3e50bc-637a-4dec-85d9-008234ff92aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:39.419981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3e50bc-637a-4dec-85d9-008234ff92aa admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:39.432177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b65d69ae-d83e-4dd0-80df-cc073d767f5c None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:39.437659 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 780/1551] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:39.437909 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b65d69ae-d83e-4dd0-80df-cc073d767f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:39.438030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7daf0-ace6-403b-a6a6-962d34f2aa49 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.438030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7daf0-ace6-403b-a6a6-962d34f2aa49 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.438030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7daf0-ace6-403b-a6a6-962d34f2aa49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.442092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 772/1552] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:39.443005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d69ae-d83e-4dd0-80df-cc073d767f5c None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.453721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d69ae-d83e-4dd0-80df-cc073d767f5c None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:39.457665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-a4e7daf0-ace6-403b-a6a6-962d34f2aa49 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:39.458922 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-a4e7daf0-ace6-403b-a6a6-962d34f2aa49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:39.459048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af95bf-f3ab-4704-928e-14ffcf64d4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:39.459638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af95bf-f3ab-4704-928e-14ffcf64d4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.459907 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af95bf-f3ab-4704-928e-14ffcf64d4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.460174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af95bf-f3ab-4704-928e-14ffcf64d4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.460901 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af95bf-f3ab-4704-928e-14ffcf64d4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:39.462584 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af95bf-f3ab-4704-928e-14ffcf64d4d4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:39.468336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 781/1553] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:39.469558 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b615be0-02b0-4e34-96b4-c097c5e777f7 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.469891 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b615be0-02b0-4e34-96b4-c097c5e777f7 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.470154 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b615be0-02b0-4e34-96b4-c097c5e777f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.475293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 773/1554] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:39.478400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af95bf-f3ab-4704-928e-14ffcf64d4d4 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.486514 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af95bf-f3ab-4704-928e-14ffcf64d4d4 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:39.490846 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-9b615be0-02b0-4e34-96b4-c097c5e777f7 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:39.491186 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c2da5-6895-47f1-b46a-ce38ead6729e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:39.491799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c2da5-6895-47f1-b46a-ce38ead6729e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.492073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c2da5-6895-47f1-b46a-ce38ead6729e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.492189 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9b615be0-02b0-4e34-96b4-c097c5e777f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:39.492367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c2da5-6895-47f1-b46a-ce38ead6729e admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.493023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c2da5-6895-47f1-b46a-ce38ead6729e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:39.493319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c2da5-6895-47f1-b46a-ce38ead6729e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:39.494369 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: WARNING py.warnings [None req-b87c2da5-6895-47f1-b46a-ce38ead6729e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:23:39.494369 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: warnings.warn(msg) Aug 01 15:23:39.494369 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: Aug 01 15:23:39.495293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c2da5-6895-47f1-b46a-ce38ead6729e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:39.500713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 782/1555] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:39.502025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8ab0c-0197-41a4-89f0-31b95beb84cd None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.502310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8ab0c-0197-41a4-89f0-31b95beb84cd None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.502626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8ab0c-0197-41a4-89f0-31b95beb84cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.511599 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 774/1556] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:39.513567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0b150-084a-41cf-b667-82df11a82ea9 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.514071 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0b150-084a-41cf-b667-82df11a82ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.514676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0b150-084a-41cf-b667-82df11a82ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.522631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-02e8ab0c-0197-41a4-89f0-31b95beb84cd None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:39.523807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-02e8ab0c-0197-41a4-89f0-31b95beb84cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:39.531817 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 783/1557] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:39.532637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8ab0c-0197-41a4-89f0-31b95beb84cd None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.534487 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-50e0b150-084a-41cf-b667-82df11a82ea9 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:39.535682 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-50e0b150-084a-41cf-b667-82df11a82ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:39.545106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8ab0c-0197-41a4-89f0-31b95beb84cd None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:39.545304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 775/1558] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:39.546740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4eb12f-1f02-403e-9193-b5eb59f12d6b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.547082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4eb12f-1f02-403e-9193-b5eb59f12d6b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.547352 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4eb12f-1f02-403e-9193-b5eb59f12d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.550637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360ed2a-00c7-4b59-b171-3884aaa4cdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:39.551533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360ed2a-00c7-4b59-b171-3884aaa4cdac admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.552004 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360ed2a-00c7-4b59-b171-3884aaa4cdac admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.552467 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360ed2a-00c7-4b59-b171-3884aaa4cdac admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.553722 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360ed2a-00c7-4b59-b171-3884aaa4cdac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:39.554817 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: WARNING py.warnings [None req-2360ed2a-00c7-4b59-b171-3884aaa4cdac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:23:39.554817 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: warnings.warn(msg) Aug 01 15:23:39.554817 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: Aug 01 15:23:39.556187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360ed2a-00c7-4b59-b171-3884aaa4cdac admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:39.568131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-1d4eb12f-1f02-403e-9193-b5eb59f12d6b None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:39.569366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1d4eb12f-1f02-403e-9193-b5eb59f12d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:39.579287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 776/1559] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:39.583494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b903a1f-427c-41a0-8586-3b1e25fbf0a3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.583784 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b903a1f-427c-41a0-8586-3b1e25fbf0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.584047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b903a1f-427c-41a0-8586-3b1e25fbf0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.587507 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 784/1560] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:39.588535 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360ed2a-00c7-4b59-b171-3884aaa4cdac admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.605285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-0b903a1f-427c-41a0-8586-3b1e25fbf0a3 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:39.606539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0b903a1f-427c-41a0-8586-3b1e25fbf0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:39.646194 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:39.652093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360ed2a-00c7-4b59-b171-3884aaa4cdac admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:39.658960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f6d65-e8c8-49d1-88b6-50dbf7254d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:39.659657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f6d65-e8c8-49d1-88b6-50dbf7254d9a admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.659939 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f6d65-e8c8-49d1-88b6-50dbf7254d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.660211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f6d65-e8c8-49d1-88b6-50dbf7254d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.660922 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f6d65-e8c8-49d1-88b6-50dbf7254d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:39.662591 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f6d65-e8c8-49d1-88b6-50dbf7254d9a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:39.666534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 777/1561] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:39.667424 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b903a1f-427c-41a0-8586-3b1e25fbf0a3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.676629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 785/1562] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:39.677460 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f6d65-e8c8-49d1-88b6-50dbf7254d9a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.680601 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b903a1f-427c-41a0-8586-3b1e25fbf0a3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:39.685499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca0175-6b91-4c45-8557-14186042dd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:39.686151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca0175-6b91-4c45-8557-14186042dd89 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.686431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca0175-6b91-4c45-8557-14186042dd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.686709 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca0175-6b91-4c45-8557-14186042dd89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.687195 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f6d65-e8c8-49d1-88b6-50dbf7254d9a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:39.687410 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca0175-6b91-4c45-8557-14186042dd89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:39.687713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca0175-6b91-4c45-8557-14186042dd89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:39.689332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebca0175-6b91-4c45-8557-14186042dd89 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:39.691002 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:39.691723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d89a4-55ca-4c4a-afdd-6ce33e369007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:39.692561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d89a4-55ca-4c4a-afdd-6ce33e369007 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.692688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d89a4-55ca-4c4a-afdd-6ce33e369007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.692917 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d89a4-55ca-4c4a-afdd-6ce33e369007 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.693536 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d89a4-55ca-4c4a-afdd-6ce33e369007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:39.695750 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d89a4-55ca-4c4a-afdd-6ce33e369007 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:39.702462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 778/1563] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:39.708762 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca0175-6b91-4c45-8557-14186042dd89 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.719916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca0175-6b91-4c45-8557-14186042dd89 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:39.725059 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 786/1564] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:39.725296 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f98179-3a3f-440e-b0c8-8f91c4dffc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:39.726064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f98179-3a3f-440e-b0c8-8f91c4dffc72 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.726346 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f98179-3a3f-440e-b0c8-8f91c4dffc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.726623 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f98179-3a3f-440e-b0c8-8f91c4dffc72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.727327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f98179-3a3f-440e-b0c8-8f91c4dffc72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:39.727556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15ba75-a107-4a2c-a601-8dc496258510 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.727659 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f98179-3a3f-440e-b0c8-8f91c4dffc72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:39.728051 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15ba75-a107-4a2c-a601-8dc496258510 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.728480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15ba75-a107-4a2c-a601-8dc496258510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.729441 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f98179-3a3f-440e-b0c8-8f91c4dffc72 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:39.742717 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 779/1565] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:39.743580 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f98179-3a3f-440e-b0c8-8f91c4dffc72 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.748952 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2d15ba75-a107-4a2c-a601-8dc496258510 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:39.751298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2d15ba75-a107-4a2c-a601-8dc496258510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:39.754026 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f98179-3a3f-440e-b0c8-8f91c4dffc72 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:39.813629 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec7f57-9e34-474c-9c98-b682a68a9c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:39.814296 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec7f57-9e34-474c-9c98-b682a68a9c4c admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.814577 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec7f57-9e34-474c-9c98-b682a68a9c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.814854 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec7f57-9e34-474c-9c98-b682a68a9c4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.815607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec7f57-9e34-474c-9c98-b682a68a9c4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:39.815866 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec7f57-9e34-474c-9c98-b682a68a9c4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:39.817491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec7f57-9e34-474c-9c98-b682a68a9c4c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:39.817712 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 787/1566] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:39.821654 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15ba75-a107-4a2c-a601-8dc496258510 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.828522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15ba75-a107-4a2c-a601-8dc496258510 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:39.835732 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 780/1567] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:39.835732 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec7f57-9e34-474c-9c98-b682a68a9c4c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.840181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d50c2-90a4-4145-9954-b5e10f247c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:39.840833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d50c2-90a4-4145-9954-b5e10f247c81 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.841109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d50c2-90a4-4145-9954-b5e10f247c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.841379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d50c2-90a4-4145-9954-b5e10f247c81 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.841846 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec7f57-9e34-474c-9c98-b682a68a9c4c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:39.842096 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d50c2-90a4-4145-9954-b5e10f247c81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:39.844100 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d50c2-90a4-4145-9954-b5e10f247c81 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:39.847389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123192a-88fd-444d-859d-b0ce80890fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:39.847724 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123192a-88fd-444d-859d-b0ce80890fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.848007 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123192a-88fd-444d-859d-b0ce80890fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.848281 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123192a-88fd-444d-859d-b0ce80890fc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.849007 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123192a-88fd-444d-859d-b0ce80890fc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:39.849313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123192a-88fd-444d-859d-b0ce80890fc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:39.850950 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123192a-88fd-444d-859d-b0ce80890fc3 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:39.865999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 781/1568] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:39.867032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123192a-88fd-444d-859d-b0ce80890fc3 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.878397 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123192a-88fd-444d-859d-b0ce80890fc3 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:39.881551 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 788/1569] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/users => generated 296 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:39.882724 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d50c2-90a4-4145-9954-b5e10f247c81 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.883040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac696a-fa0a-4049-8458-d16cdc6e827d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:39.883736 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac696a-fa0a-4049-8458-d16cdc6e827d admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.884021 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac696a-fa0a-4049-8458-d16cdc6e827d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.884346 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac696a-fa0a-4049-8458-d16cdc6e827d admin admin] PATH_INFO: `/v3/projects/7c2c70a52022408fbc58c1a969f721b9/users/5f23c0f21bf54515b8cdc03d415213c5/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.893508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d50c2-90a4-4145-9954-b5e10f247c81 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:39.900402 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac696a-fa0a-4049-8458-d16cdc6e827d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c2c70a52022408fbc58c1a969f721b9, user_id=5f23c0f21bf54515b8cdc03d415213c5, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:39.903074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee32da-136d-4f27-a05b-cb766489b183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:39.903227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac696a-fa0a-4049-8458-d16cdc6e827d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:39.903707 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee32da-136d-4f27-a05b-cb766489b183 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:39.903984 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee32da-136d-4f27-a05b-cb766489b183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:39.906478 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee32da-136d-4f27-a05b-cb766489b183 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:39.907268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee32da-136d-4f27-a05b-cb766489b183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:39.910033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee32da-136d-4f27-a05b-cb766489b183 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:39.917455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 782/1570] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:39 2022] PUT /identity/v3/projects/7c2c70a52022408fbc58c1a969f721b9/users/5f23c0f21bf54515b8cdc03d415213c5/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:39.918408 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac696a-fa0a-4049-8458-d16cdc6e827d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.947027 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 789/1571] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:39.950787 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee32da-136d-4f27-a05b-cb766489b183 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:39.989107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac696a-fa0a-4049-8458-d16cdc6e827d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.012305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee32da-136d-4f27-a05b-cb766489b183 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.017479 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd3fb6-3fb3-4b28-b00c-e04848913599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.018134 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd3fb6-3fb3-4b28-b00c-e04848913599 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.018410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd3fb6-3fb3-4b28-b00c-e04848913599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.018678 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd3fb6-3fb3-4b28-b00c-e04848913599 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.019337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd3fb6-3fb3-4b28-b00c-e04848913599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.021296 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd3fb6-3fb3-4b28-b00c-e04848913599 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.032546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:40.056336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587a222-f465-4b45-a41a-01adb2b9a718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.057007 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587a222-f465-4b45-a41a-01adb2b9a718 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.057312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587a222-f465-4b45-a41a-01adb2b9a718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.057650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587a222-f465-4b45-a41a-01adb2b9a718 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.058168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 790/1572] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:40.058525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587a222-f465-4b45-a41a-01adb2b9a718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.059660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: WARNING py.warnings [None req-0587a222-f465-4b45-a41a-01adb2b9a718 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:23:40.059660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: warnings.warn(msg) Aug 01 15:23:40.059660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: Aug 01 15:23:40.059921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd3fb6-3fb3-4b28-b00c-e04848913599 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.060887 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587a222-f465-4b45-a41a-01adb2b9a718 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.083494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:40.088348 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 783/1573] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:23:39 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:40.088348 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587a222-f465-4b45-a41a-01adb2b9a718 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.096925 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587a222-f465-4b45-a41a-01adb2b9a718 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.133062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd3fb6-3fb3-4b28-b00c-e04848913599 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.138055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ceb2a7-4d9b-4d4b-b6e8-c4a3f85bb60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.138702 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceb2a7-4d9b-4d4b-b6e8-c4a3f85bb60b admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.138987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceb2a7-4d9b-4d4b-b6e8-c4a3f85bb60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.139291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ceb2a7-4d9b-4d4b-b6e8-c4a3f85bb60b admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.139997 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ceb2a7-4d9b-4d4b-b6e8-c4a3f85bb60b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:40.140304 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ceb2a7-4d9b-4d4b-b6e8-c4a3f85bb60b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.141985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ceb2a7-4d9b-4d4b-b6e8-c4a3f85bb60b admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.154583 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 791/1574] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:40.155153 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ceb2a7-4d9b-4d4b-b6e8-c4a3f85bb60b admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.162903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517877c-d666-4562-86db-6c688f483edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.163578 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517877c-d666-4562-86db-6c688f483edf admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.163855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517877c-d666-4562-86db-6c688f483edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.164372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517877c-d666-4562-86db-6c688f483edf admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.165051 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517877c-d666-4562-86db-6c688f483edf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.167200 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517877c-d666-4562-86db-6c688f483edf admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.169166 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ceb2a7-4d9b-4d4b-b6e8-c4a3f85bb60b admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.173767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646af56-2d0e-43ce-8c7c-13359e948922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.174416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646af56-2d0e-43ce-8c7c-13359e948922 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.174698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646af56-2d0e-43ce-8c7c-13359e948922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.174971 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646af56-2d0e-43ce-8c7c-13359e948922 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.175637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646af56-2d0e-43ce-8c7c-13359e948922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.177720 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646af56-2d0e-43ce-8c7c-13359e948922 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.196588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 784/1575] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:23:40 2022] POST /identity/v3/projects => generated 403 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:40.197520 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517877c-d666-4562-86db-6c688f483edf admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.213476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 792/1576] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:23:40 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:40.213476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d3277-1d08-48b1-a2fa-4f16aa2dd63b None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.213476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d3277-1d08-48b1-a2fa-4f16aa2dd63b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.213476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d3277-1d08-48b1-a2fa-4f16aa2dd63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.246161 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-1f7d3277-1d08-48b1-a2fa-4f16aa2dd63b None None] MFA Rules not processed for user `5f23c0f21bf54515b8cdc03d415213c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:40.247400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1f7d3277-1d08-48b1-a2fa-4f16aa2dd63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:40.276903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517877c-d666-4562-86db-6c688f483edf admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.313615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 793/1577] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:40.314630 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d3277-1d08-48b1-a2fa-4f16aa2dd63b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.339488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37eab27-1a1b-4d77-873f-43e8cf30c75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.340599 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37eab27-1a1b-4d77-873f-43e8cf30c75f admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.340907 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37eab27-1a1b-4d77-873f-43e8cf30c75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.341183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37eab27-1a1b-4d77-873f-43e8cf30c75f admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.341968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37eab27-1a1b-4d77-873f-43e8cf30c75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.344152 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37eab27-1a1b-4d77-873f-43e8cf30c75f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.356112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 785/1578] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:40.356958 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37eab27-1a1b-4d77-873f-43e8cf30c75f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.362664 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:40.369373 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37eab27-1a1b-4d77-873f-43e8cf30c75f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.374875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb422361-88c5-4534-8dd8-0b33d69fe24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.375509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb422361-88c5-4534-8dd8-0b33d69fe24c admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.375785 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb422361-88c5-4534-8dd8-0b33d69fe24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.376056 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb422361-88c5-4534-8dd8-0b33d69fe24c admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.376723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb422361-88c5-4534-8dd8-0b33d69fe24c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.378733 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb422361-88c5-4534-8dd8-0b33d69fe24c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.383660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d3277-1d08-48b1-a2fa-4f16aa2dd63b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.388262 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34cbca-b7ac-4ae2-b3aa-f8cfbf01737a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.388866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34cbca-b7ac-4ae2-b3aa-f8cfbf01737a admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.389166 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34cbca-b7ac-4ae2-b3aa-f8cfbf01737a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.389436 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34cbca-b7ac-4ae2-b3aa-f8cfbf01737a admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.390129 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34cbca-b7ac-4ae2-b3aa-f8cfbf01737a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.392248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34cbca-b7ac-4ae2-b3aa-f8cfbf01737a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.402096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 786/1579] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:23:40 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:40.402987 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb422361-88c5-4534-8dd8-0b33d69fe24c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.414761 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 794/1580] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:23:40 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:40.415839 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34cbca-b7ac-4ae2-b3aa-f8cfbf01737a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.417812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb422361-88c5-4534-8dd8-0b33d69fe24c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.422665 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a6f10-63f2-4ca5-b4be-b58f758f719b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.423291 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a6f10-63f2-4ca5-b4be-b58f758f719b admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.423571 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a6f10-63f2-4ca5-b4be-b58f758f719b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.423902 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a6f10-63f2-4ca5-b4be-b58f758f719b admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.424606 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a6f10-63f2-4ca5-b4be-b58f758f719b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.426294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a6f10-63f2-4ca5-b4be-b58f758f719b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.426526 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34cbca-b7ac-4ae2-b3aa-f8cfbf01737a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.431793 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fc22a-779f-4799-a937-16a1c1ffa4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.432451 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fc22a-779f-4799-a937-16a1c1ffa4c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.432790 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fc22a-779f-4799-a937-16a1c1ffa4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.433126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fc22a-779f-4799-a937-16a1c1ffa4c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.433807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fc22a-779f-4799-a937-16a1c1ffa4c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.435720 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fc22a-779f-4799-a937-16a1c1ffa4c0 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.437640 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 787/1581] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:40.438594 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a6f10-63f2-4ca5-b4be-b58f758f719b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.440544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:40.448668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a6f10-63f2-4ca5-b4be-b58f758f719b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.453994 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0701bc-9265-4a47-b11a-6b57d6a52f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.455152 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0701bc-9265-4a47-b11a-6b57d6a52f98 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.455152 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0701bc-9265-4a47-b11a-6b57d6a52f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.455152 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0701bc-9265-4a47-b11a-6b57d6a52f98 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.455802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0701bc-9265-4a47-b11a-6b57d6a52f98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.457876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0701bc-9265-4a47-b11a-6b57d6a52f98 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.468345 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 795/1582] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:23:40 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:40.469469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fc22a-779f-4799-a937-16a1c1ffa4c0 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.488266 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 788/1583] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:23:40 2022] POST /identity/v3/users => generated 302 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:40.488266 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0701bc-9265-4a47-b11a-6b57d6a52f98 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.542402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fc22a-779f-4799-a937-16a1c1ffa4c0 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.544031 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0701bc-9265-4a47-b11a-6b57d6a52f98 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.547798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946d221-b551-41bf-8763-d5d2447a5a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.548438 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946d221-b551-41bf-8763-d5d2447a5a3f admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.548717 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946d221-b551-41bf-8763-d5d2447a5a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.548985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946d221-b551-41bf-8763-d5d2447a5a3f admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.549657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946d221-b551-41bf-8763-d5d2447a5a3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.551649 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946d221-b551-41bf-8763-d5d2447a5a3f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.580306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 796/1584] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:23:40 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:40.580306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946d221-b551-41bf-8763-d5d2447a5a3f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.588662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946d221-b551-41bf-8763-d5d2447a5a3f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.596064 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668947e-3ed4-4be9-bed2-133cbc512b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.596719 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668947e-3ed4-4be9-bed2-133cbc512b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.596998 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668947e-3ed4-4be9-bed2-133cbc512b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.597273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668947e-3ed4-4be9-bed2-133cbc512b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.598032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668947e-3ed4-4be9-bed2-133cbc512b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.599642 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd2687-ea32-4965-9a3f-7f0a2514de73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.599796 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668947e-3ed4-4be9-bed2-133cbc512b95 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.600286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd2687-ea32-4965-9a3f-7f0a2514de73 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.600565 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd2687-ea32-4965-9a3f-7f0a2514de73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.600866 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd2687-ea32-4965-9a3f-7f0a2514de73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.611289 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 797/1585] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:40.612298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668947e-3ed4-4be9-bed2-133cbc512b95 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.622723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668947e-3ed4-4be9-bed2-133cbc512b95 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.628436 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c322a-9077-4ccd-869f-25aa6faf08f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.629071 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c322a-9077-4ccd-869f-25aa6faf08f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.629349 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c322a-9077-4ccd-869f-25aa6faf08f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.630053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c322a-9077-4ccd-869f-25aa6faf08f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.630766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c322a-9077-4ccd-869f-25aa6faf08f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.632874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c322a-9077-4ccd-869f-25aa6faf08f7 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.644155 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 798/1586] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:40.645002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c322a-9077-4ccd-869f-25aa6faf08f7 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.657174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c322a-9077-4ccd-869f-25aa6faf08f7 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.661799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41304f-6570-421f-a9c6-7df86f2876d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.662421 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41304f-6570-421f-a9c6-7df86f2876d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.662700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41304f-6570-421f-a9c6-7df86f2876d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.662971 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41304f-6570-421f-a9c6-7df86f2876d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.663674 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41304f-6570-421f-a9c6-7df86f2876d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.665494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41304f-6570-421f-a9c6-7df86f2876d0 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.668900 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd2687-ea32-4965-9a3f-7f0a2514de73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.670429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd2687-ea32-4965-9a3f-7f0a2514de73 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.675879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 799/1587] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:40.676650 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41304f-6570-421f-a9c6-7df86f2876d0 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.679973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 789/1588] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:40.680940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd2687-ea32-4965-9a3f-7f0a2514de73 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.687131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41304f-6570-421f-a9c6-7df86f2876d0 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.691666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd2687-ea32-4965-9a3f-7f0a2514de73 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.691960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca32512-9a67-4424-b344-1df0e54f00d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.692583 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca32512-9a67-4424-b344-1df0e54f00d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.692862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca32512-9a67-4424-b344-1df0e54f00d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.693136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca32512-9a67-4424-b344-1df0e54f00d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.693857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca32512-9a67-4424-b344-1df0e54f00d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.695539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca32512-9a67-4424-b344-1df0e54f00d5 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.696732 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7a123-4d35-45ce-9f49-6451e07a6012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.697432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7a123-4d35-45ce-9f49-6451e07a6012 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.697797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7a123-4d35-45ce-9f49-6451e07a6012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.698146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7a123-4d35-45ce-9f49-6451e07a6012 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.698888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7a123-4d35-45ce-9f49-6451e07a6012 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.700828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7a123-4d35-45ce-9f49-6451e07a6012 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.709869 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 800/1589] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:40.709869 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca32512-9a67-4424-b344-1df0e54f00d5 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.720490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca32512-9a67-4424-b344-1df0e54f00d5 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.725665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be880d1-80ba-4af8-934f-b47dcf8c1b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.726112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 790/1590] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:23:40 2022] POST /identity/v3/users => generated 283 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:40.727498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7a123-4d35-45ce-9f49-6451e07a6012 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.729855 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be880d1-80ba-4af8-934f-b47dcf8c1b5c admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.730145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be880d1-80ba-4af8-934f-b47dcf8c1b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.730419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be880d1-80ba-4af8-934f-b47dcf8c1b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.731224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be880d1-80ba-4af8-934f-b47dcf8c1b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.732855 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be880d1-80ba-4af8-934f-b47dcf8c1b5c admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.739600 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7a123-4d35-45ce-9f49-6451e07a6012 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.744648 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc682164-831f-4a0f-a2b6-8cc292226e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.744821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 801/1591] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:40.745056 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc682164-831f-4a0f-a2b6-8cc292226e9b admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.745342 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc682164-831f-4a0f-a2b6-8cc292226e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.745630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc682164-831f-4a0f-a2b6-8cc292226e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.746520 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc682164-831f-4a0f-a2b6-8cc292226e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.748207 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc682164-831f-4a0f-a2b6-8cc292226e9b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.748500 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be880d1-80ba-4af8-934f-b47dcf8c1b5c admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.751478 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:40.760089 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be880d1-80ba-4af8-934f-b47dcf8c1b5c admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.764608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 791/1592] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:40.765562 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efe3f7-f370-4f70-b6c9-d7f8a972ac0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.765698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc682164-831f-4a0f-a2b6-8cc292226e9b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.766327 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efe3f7-f370-4f70-b6c9-d7f8a972ac0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.766565 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efe3f7-f370-4f70-b6c9-d7f8a972ac0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.766839 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efe3f7-f370-4f70-b6c9-d7f8a972ac0e admin admin] PATH_INFO: `/v3/projects/a359662d2a444976af8b1b9cc351edb9/users/6f852249c66a4f8986d7a63a957ec91c/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.777480 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc682164-831f-4a0f-a2b6-8cc292226e9b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.782784 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a1273-2943-4e9a-a883-001048a6ed5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.783410 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a1273-2943-4e9a-a883-001048a6ed5c admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.783694 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a1273-2943-4e9a-a883-001048a6ed5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.784099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a1273-2943-4e9a-a883-001048a6ed5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.784275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efe3f7-f370-4f70-b6c9-d7f8a972ac0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a359662d2a444976af8b1b9cc351edb9, user_id=6f852249c66a4f8986d7a63a957ec91c, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.784799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a1273-2943-4e9a-a883-001048a6ed5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.785335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: WARNING py.warnings [None req-a1efe3f7-f370-4f70-b6c9-d7f8a972ac0e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:23:40.785335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: warnings.warn(msg) Aug 01 15:23:40.785335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: Aug 01 15:23:40.786592 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a1273-2943-4e9a-a883-001048a6ed5c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.787306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efe3f7-f370-4f70-b6c9-d7f8a972ac0e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.798431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 792/1593] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:40.802499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a1273-2943-4e9a-a883-001048a6ed5c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.804207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 802/1594] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:40 2022] PUT /identity/v3/projects/a359662d2a444976af8b1b9cc351edb9/users/6f852249c66a4f8986d7a63a957ec91c/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:40.805052 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efe3f7-f370-4f70-b6c9-d7f8a972ac0e admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.816488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a1273-2943-4e9a-a883-001048a6ed5c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.876177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:40.882387 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07684543-7f87-4cea-9961-1326b55ce253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.883145 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07684543-7f87-4cea-9961-1326b55ce253 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.883530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07684543-7f87-4cea-9961-1326b55ce253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.883900 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07684543-7f87-4cea-9961-1326b55ce253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.884812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07684543-7f87-4cea-9961-1326b55ce253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.884937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efe3f7-f370-4f70-b6c9-d7f8a972ac0e admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.886606 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07684543-7f87-4cea-9961-1326b55ce253 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.890917 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b634372-d576-495a-8c28-137d49b9ff7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.891606 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b634372-d576-495a-8c28-137d49b9ff7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.891885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b634372-d576-495a-8c28-137d49b9ff7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.892158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b634372-d576-495a-8c28-137d49b9ff7f admin admin] PATH_INFO: `/v3/projects/5400c4df102f43398a278247c4431ef6/users/c60ee64ca18e4930a14e3e17bc979ff7/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.901048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 793/1595] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:40.901048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07684543-7f87-4cea-9961-1326b55ce253 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.910274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b634372-d576-495a-8c28-137d49b9ff7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5400c4df102f43398a278247c4431ef6, user_id=c60ee64ca18e4930a14e3e17bc979ff7, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.910979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07684543-7f87-4cea-9961-1326b55ce253 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:40.913345 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b634372-d576-495a-8c28-137d49b9ff7f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.917825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4aa58d-e4ff-4797-bc93-aaa456d2ac3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:40.919235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4aa58d-e4ff-4797-bc93-aaa456d2ac3e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:40.919566 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4aa58d-e4ff-4797-bc93-aaa456d2ac3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:40.919851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4aa58d-e4ff-4797-bc93-aaa456d2ac3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:40.920751 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4aa58d-e4ff-4797-bc93-aaa456d2ac3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:40.922569 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4aa58d-e4ff-4797-bc93-aaa456d2ac3e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:40.934989 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 803/1596] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:40 2022] PUT /identity/v3/projects/5400c4df102f43398a278247c4431ef6/users/c60ee64ca18e4930a14e3e17bc979ff7/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:40.934989 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b634372-d576-495a-8c28-137d49b9ff7f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.939453 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 794/1597] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:40.943141 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4aa58d-e4ff-4797-bc93-aaa456d2ac3e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:40.954805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4aa58d-e4ff-4797-bc93-aaa456d2ac3e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.005264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b634372-d576-495a-8c28-137d49b9ff7f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.010418 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14bf4a-5807-46a0-890c-9487a7eba11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.011190 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14bf4a-5807-46a0-890c-9487a7eba11f admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.011525 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14bf4a-5807-46a0-890c-9487a7eba11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.011789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14bf4a-5807-46a0-890c-9487a7eba11f admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.012410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14bf4a-5807-46a0-890c-9487a7eba11f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.014207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14bf4a-5807-46a0-890c-9487a7eba11f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.024446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea4d2f-7604-434d-9912-cc686c084798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.025605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 804/1598] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:41.026308 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea4d2f-7604-434d-9912-cc686c084798 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.026509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14bf4a-5807-46a0-890c-9487a7eba11f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.026947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea4d2f-7604-434d-9912-cc686c084798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.027508 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea4d2f-7604-434d-9912-cc686c084798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.028435 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea4d2f-7604-434d-9912-cc686c084798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.030552 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea4d2f-7604-434d-9912-cc686c084798 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.038845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14bf4a-5807-46a0-890c-9487a7eba11f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.043795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4676f80-0ef6-46f1-b0ad-a7d132f3ab6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.047604 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 795/1599] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:41.047765 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4676f80-0ef6-46f1-b0ad-a7d132f3ab6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.047765 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4676f80-0ef6-46f1-b0ad-a7d132f3ab6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.047765 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4676f80-0ef6-46f1-b0ad-a7d132f3ab6b admin admin] PATH_INFO: `/v3/projects/3b8e7cb5fc4a4e598d27e51122265bda/users/93d86ae4501e43d59d172d2e7acfdd54/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.048082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e506d4-5d72-4f87-bb1b-ad657239e894 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.048082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e506d4-5d72-4f87-bb1b-ad657239e894 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.048082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e506d4-5d72-4f87-bb1b-ad657239e894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.061393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4676f80-0ef6-46f1-b0ad-a7d132f3ab6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b8e7cb5fc4a4e598d27e51122265bda, user_id=93d86ae4501e43d59d172d2e7acfdd54, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.064099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4676f80-0ef6-46f1-b0ad-a7d132f3ab6b admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.071766 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d3e506d4-5d72-4f87-bb1b-ad657239e894 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:41.073064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d3e506d4-5d72-4f87-bb1b-ad657239e894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:41.080316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 805/1600] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:41 2022] PUT /identity/v3/projects/3b8e7cb5fc4a4e598d27e51122265bda/users/93d86ae4501e43d59d172d2e7acfdd54/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:41.081954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4676f80-0ef6-46f1-b0ad-a7d132f3ab6b admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.146604 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 796/1601] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:41.148352 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2cd4a-a9ce-47d0-9a96-e76e50af0618 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.148588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2cd4a-a9ce-47d0-9a96-e76e50af0618 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.148845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2cd4a-a9ce-47d0-9a96-e76e50af0618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.161803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4676f80-0ef6-46f1-b0ad-a7d132f3ab6b admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.166942 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24925ce-ee84-4330-aff9-2608d292de0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.167679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24925ce-ee84-4330-aff9-2608d292de0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.168018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24925ce-ee84-4330-aff9-2608d292de0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.168260 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24925ce-ee84-4330-aff9-2608d292de0b admin admin] PATH_INFO: `/v3/projects/9d47732c77c34b63a94b3f682f420416/users/d79f95d9699b4b0393e14a9212986ad5/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.170901 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-48e2cd4a-a9ce-47d0-9a96-e76e50af0618 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:41.172172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-48e2cd4a-a9ce-47d0-9a96-e76e50af0618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:41.190184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 797/1602] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:41.190184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2cd4a-a9ce-47d0-9a96-e76e50af0618 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.190184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:41.190462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24925ce-ee84-4330-aff9-2608d292de0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d47732c77c34b63a94b3f682f420416, user_id=d79f95d9699b4b0393e14a9212986ad5, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.193825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24925ce-ee84-4330-aff9-2608d292de0b admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.194815 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2cd4a-a9ce-47d0-9a96-e76e50af0618 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.202460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae01bce-cb94-4f0e-93f7-42e0204f50e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.203423 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae01bce-cb94-4f0e-93f7-42e0204f50e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.203822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae01bce-cb94-4f0e-93f7-42e0204f50e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.204311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae01bce-cb94-4f0e-93f7-42e0204f50e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.205300 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae01bce-cb94-4f0e-93f7-42e0204f50e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.207510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae01bce-cb94-4f0e-93f7-42e0204f50e6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.213082 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 806/1603] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:41 2022] PUT /identity/v3/projects/9d47732c77c34b63a94b3f682f420416/users/d79f95d9699b4b0393e14a9212986ad5/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:41.214529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24925ce-ee84-4330-aff9-2608d292de0b admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.221817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 798/1604] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:41.223158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae01bce-cb94-4f0e-93f7-42e0204f50e6 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.235583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae01bce-cb94-4f0e-93f7-42e0204f50e6 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.271056 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:41.315210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24925ce-ee84-4330-aff9-2608d292de0b admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.320696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb6200-62c8-4e4b-9cd1-ba8a95d6c40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.321488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb6200-62c8-4e4b-9cd1-ba8a95d6c40d admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.321862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb6200-62c8-4e4b-9cd1-ba8a95d6c40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.322206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb6200-62c8-4e4b-9cd1-ba8a95d6c40d admin admin] PATH_INFO: `/v3/projects/2632844dad0b49f4beb8728d99e19e2a/users/d7efb76dbe374678b5e20f847f1a3dcb/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.331599 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752ed50-4f07-49fb-b096-ac1f9c2fc5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.332537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752ed50-4f07-49fb-b096-ac1f9c2fc5af admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.333140 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752ed50-4f07-49fb-b096-ac1f9c2fc5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.333500 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752ed50-4f07-49fb-b096-ac1f9c2fc5af admin admin] PATH_INFO: `/v3/projects/b13cfc3643554bc785b182b3699058e2/users/e6945b8e606f40f1b79bd4aab96a7f1a/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.342842 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb6200-62c8-4e4b-9cd1-ba8a95d6c40d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2632844dad0b49f4beb8728d99e19e2a, user_id=d7efb76dbe374678b5e20f847f1a3dcb, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.346016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb6200-62c8-4e4b-9cd1-ba8a95d6c40d admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.353038 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752ed50-4f07-49fb-b096-ac1f9c2fc5af admin admin] RBAC: Authorizing `identity:create_grant(project_id=b13cfc3643554bc785b182b3699058e2, user_id=e6945b8e606f40f1b79bd4aab96a7f1a, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.356340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752ed50-4f07-49fb-b096-ac1f9c2fc5af admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.362068 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 807/1605] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:41 2022] PUT /identity/v3/projects/2632844dad0b49f4beb8728d99e19e2a/users/d7efb76dbe374678b5e20f847f1a3dcb/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:41.363380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1ebdd-7d55-4dcd-a4d9-f1768a62c7ef None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.363701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1ebdd-7d55-4dcd-a4d9-f1768a62c7ef None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.363974 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1ebdd-7d55-4dcd-a4d9-f1768a62c7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.381513 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 799/1606] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:41 2022] PUT /identity/v3/projects/b13cfc3643554bc785b182b3699058e2/users/e6945b8e606f40f1b79bd4aab96a7f1a/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:41.385854 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752ed50-4f07-49fb-b096-ac1f9c2fc5af admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.386659 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-05c1ebdd-7d55-4dcd-a4d9-f1768a62c7ef None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:41.388579 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-05c1ebdd-7d55-4dcd-a4d9-f1768a62c7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:41.444420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752ed50-4f07-49fb-b096-ac1f9c2fc5af admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.448092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76acfb3-3f28-4109-9fd1-33ae69edab97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.448922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76acfb3-3f28-4109-9fd1-33ae69edab97 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.449314 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76acfb3-3f28-4109-9fd1-33ae69edab97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.449587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76acfb3-3f28-4109-9fd1-33ae69edab97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.457917 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 808/1607] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:41.459148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1ebdd-7d55-4dcd-a4d9-f1768a62c7ef None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.459338 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76acfb3-3f28-4109-9fd1-33ae69edab97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.460786 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76acfb3-3f28-4109-9fd1-33ae69edab97 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.470900 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1ebdd-7d55-4dcd-a4d9-f1768a62c7ef None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.471654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 800/1608] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:41.472925 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d09b6-a9e8-4771-90b4-0af01471caa3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.473207 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d09b6-a9e8-4771-90b4-0af01471caa3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.473484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d09b6-a9e8-4771-90b4-0af01471caa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.475396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910a7e6-3783-412b-95b3-0af91d0e7d4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.476035 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910a7e6-3783-412b-95b3-0af91d0e7d4e service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.476316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910a7e6-3783-412b-95b3-0af91d0e7d4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.476592 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910a7e6-3783-412b-95b3-0af91d0e7d4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.487625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910a7e6-3783-412b-95b3-0af91d0e7d4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.488802 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910a7e6-3783-412b-95b3-0af91d0e7d4e service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.494317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2b6d09b6-a9e8-4771-90b4-0af01471caa3 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:41.495557 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2b6d09b6-a9e8-4771-90b4-0af01471caa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:41.500072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 809/1609] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:41.501015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910a7e6-3783-412b-95b3-0af91d0e7d4e service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.505026 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 801/1610] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:41.506178 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda323a-d690-4234-a6c4-8b3e79f09827 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.506460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda323a-d690-4234-a6c4-8b3e79f09827 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.506730 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda323a-d690-4234-a6c4-8b3e79f09827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.514363 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910a7e6-3783-412b-95b3-0af91d0e7d4e service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.520465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367599b-5b73-4b05-8b82-9e03b2165f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.521681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367599b-5b73-4b05-8b82-9e03b2165f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.522037 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367599b-5b73-4b05-8b82-9e03b2165f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.522314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367599b-5b73-4b05-8b82-9e03b2165f1c admin admin] PATH_INFO: `/v3/projects/b2de96a47291485e983fd527a03b2e27/users/837717dfd6674ca9ba8da95fe7df3eb4/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.539079 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-0eda323a-d690-4234-a6c4-8b3e79f09827 None None] MFA Rules not processed for user `d7efb76dbe374678b5e20f847f1a3dcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:41.540351 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0eda323a-d690-4234-a6c4-8b3e79f09827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:41.546420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367599b-5b73-4b05-8b82-9e03b2165f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2de96a47291485e983fd527a03b2e27, user_id=837717dfd6674ca9ba8da95fe7df3eb4, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.551800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367599b-5b73-4b05-8b82-9e03b2165f1c admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.570772 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 810/1611] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:41 2022] PUT /identity/v3/projects/b2de96a47291485e983fd527a03b2e27/users/837717dfd6674ca9ba8da95fe7df3eb4/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:41.572158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a5ed1-cad9-4b59-bcf8-23c78c46b992 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.572442 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a5ed1-cad9-4b59-bcf8-23c78c46b992 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.572717 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a5ed1-cad9-4b59-bcf8-23c78c46b992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.601200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-d72a5ed1-cad9-4b59-bcf8-23c78c46b992 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:41.602467 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d72a5ed1-cad9-4b59-bcf8-23c78c46b992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:41.643957 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:41.651994 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 802/1612] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:41.655421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda323a-d690-4234-a6c4-8b3e79f09827 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.671696 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda323a-d690-4234-a6c4-8b3e79f09827 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.674440 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 811/1613] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:41.678539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a5ed1-cad9-4b59-bcf8-23c78c46b992 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.712772 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:41.745911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb039e-c014-4201-ba1a-4fc76b7fe9ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.746667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb039e-c014-4201-ba1a-4fc76b7fe9ec service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.747045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb039e-c014-4201-ba1a-4fc76b7fe9ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.747374 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb039e-c014-4201-ba1a-4fc76b7fe9ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.759393 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb039e-c014-4201-ba1a-4fc76b7fe9ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.762128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb039e-c014-4201-ba1a-4fc76b7fe9ec service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.767082 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a5ed1-cad9-4b59-bcf8-23c78c46b992 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.771454 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955b024-6b03-486c-b94a-065798b8577e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.772490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955b024-6b03-486c-b94a-065798b8577e service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.772792 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955b024-6b03-486c-b94a-065798b8577e service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.773078 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955b024-6b03-486c-b94a-065798b8577e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.774087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 803/1614] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:41.775455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af9461-8266-4be8-a0b9-71e9511f1ef4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.775738 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af9461-8266-4be8-a0b9-71e9511f1ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.776007 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af9461-8266-4be8-a0b9-71e9511f1ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.785958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955b024-6b03-486c-b94a-065798b8577e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.787552 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955b024-6b03-486c-b94a-065798b8577e service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.802116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 812/1615] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:41.803188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955b024-6b03-486c-b94a-065798b8577e service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.814828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955b024-6b03-486c-b94a-065798b8577e service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.818018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-92af9461-8266-4be8-a0b9-71e9511f1ef4 None None] MFA Rules not processed for user `837717dfd6674ca9ba8da95fe7df3eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:41.819848 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-92af9461-8266-4be8-a0b9-71e9511f1ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:41.820440 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af974d0-e7ea-4727-8e14-f691b618a2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.821535 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af974d0-e7ea-4727-8e14-f691b618a2cc admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.821705 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af974d0-e7ea-4727-8e14-f691b618a2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.822301 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af974d0-e7ea-4727-8e14-f691b618a2cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:41.822971 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af974d0-e7ea-4727-8e14-f691b618a2cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:41.825397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af974d0-e7ea-4727-8e14-f691b618a2cc admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:41.872171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 813/1616] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:23:41 2022] POST /identity/v3/projects => generated 403 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:41.873411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af974d0-e7ea-4727-8e14-f691b618a2cc admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.954344 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 804/1617] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:41.955897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af9461-8266-4be8-a0b9-71e9511f1ef4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:41.970552 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af974d0-e7ea-4727-8e14-f691b618a2cc admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.975923 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af9461-8266-4be8-a0b9-71e9511f1ef4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:41.982698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbeaba4-c693-40fb-9129-9e0cced04ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:41.982698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbeaba4-c693-40fb-9129-9e0cced04ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:41.982698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbeaba4-c693-40fb-9129-9e0cced04ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:41.995936 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbeaba4-c693-40fb-9129-9e0cced04ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.049963 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ebd5c-0b8c-44e3-a412-13315a63a66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.050593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ebd5c-0b8c-44e3-a412-13315a63a66a admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.050869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ebd5c-0b8c-44e3-a412-13315a63a66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.051151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ebd5c-0b8c-44e3-a412-13315a63a66a admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.051890 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ebd5c-0b8c-44e3-a412-13315a63a66a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.054456 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ebd5c-0b8c-44e3-a412-13315a63a66a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.089286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 805/1618] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:23:41 2022] POST /identity/v3/users => generated 308 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:42.090599 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ebd5c-0b8c-44e3-a412-13315a63a66a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.103519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ebd5c-0b8c-44e3-a412-13315a63a66a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.111051 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f63b8-67a7-45fc-9048-cb34ac00bd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.112431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f63b8-67a7-45fc-9048-cb34ac00bd3d admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.112837 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f63b8-67a7-45fc-9048-cb34ac00bd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.113290 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f63b8-67a7-45fc-9048-cb34ac00bd3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.113406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbeaba4-c693-40fb-9129-9e0cced04ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.114265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f63b8-67a7-45fc-9048-cb34ac00bd3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.115218 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbeaba4-c693-40fb-9129-9e0cced04ed8 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.117804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f63b8-67a7-45fc-9048-cb34ac00bd3d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.128897 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 814/1619] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:42.131048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbeaba4-c693-40fb-9129-9e0cced04ed8 service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.144688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbeaba4-c693-40fb-9129-9e0cced04ed8 service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.154105 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70dade3-f7d2-4b47-a965-0c02502c71f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.154105 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70dade3-f7d2-4b47-a965-0c02502c71f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.154105 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70dade3-f7d2-4b47-a965-0c02502c71f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.156833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70dade3-f7d2-4b47-a965-0c02502c71f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.156833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70dade3-f7d2-4b47-a965-0c02502c71f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.159377 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70dade3-f7d2-4b47-a965-0c02502c71f0 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.165669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:42.175511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 815/1620] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:42.175925 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 806/1621] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:23:42 2022] POST /identity/v3/projects => generated 365 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:42.179825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f63b8-67a7-45fc-9048-cb34ac00bd3d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.180001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70dade3-f7d2-4b47-a965-0c02502c71f0 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.191495 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70dade3-f7d2-4b47-a965-0c02502c71f0 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.244740 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:42.272208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db9c2a-9fa9-461e-b10b-546e24489092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.274835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db9c2a-9fa9-461e-b10b-546e24489092 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.275352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db9c2a-9fa9-461e-b10b-546e24489092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.277390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db9c2a-9fa9-461e-b10b-546e24489092 admin admin] PATH_INFO: `/v3/projects/5e111ec2c8324461ad7741856f130b9f/users/7406519ab8f843078f40a7da578b4390/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.285146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f63b8-67a7-45fc-9048-cb34ac00bd3d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.290851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b393ee9-0de8-43f3-a649-3f43cccabf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.292462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b393ee9-0de8-43f3-a649-3f43cccabf6a admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.292980 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b393ee9-0de8-43f3-a649-3f43cccabf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.293851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b393ee9-0de8-43f3-a649-3f43cccabf6a admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.301421 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db9c2a-9fa9-461e-b10b-546e24489092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e111ec2c8324461ad7741856f130b9f, user_id=7406519ab8f843078f40a7da578b4390, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.305624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db9c2a-9fa9-461e-b10b-546e24489092 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.309012 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b393ee9-0de8-43f3-a649-3f43cccabf6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.312566 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b393ee9-0de8-43f3-a649-3f43cccabf6a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.343078 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 816/1622] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:42 2022] PUT /identity/v3/projects/5e111ec2c8324461ad7741856f130b9f/users/7406519ab8f843078f40a7da578b4390/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:42.348997 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db9c2a-9fa9-461e-b10b-546e24489092 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.371296 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 807/1623] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:23:42 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:42.376596 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b393ee9-0de8-43f3-a649-3f43cccabf6a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.398772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b393ee9-0de8-43f3-a649-3f43cccabf6a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.495575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db9c2a-9fa9-461e-b10b-546e24489092 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.501314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142ba19-7509-40f5-8ce5-5c9081184e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.502194 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142ba19-7509-40f5-8ce5-5c9081184e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.502526 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142ba19-7509-40f5-8ce5-5c9081184e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.502805 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142ba19-7509-40f5-8ce5-5c9081184e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.503525 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142ba19-7509-40f5-8ce5-5c9081184e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.505328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142ba19-7509-40f5-8ce5-5c9081184e03 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.521802 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 817/1624] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:42.523528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142ba19-7509-40f5-8ce5-5c9081184e03 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.538391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142ba19-7509-40f5-8ce5-5c9081184e03 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.542866 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb92cab-5996-44c5-97ef-633448d5d680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.543619 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb92cab-5996-44c5-97ef-633448d5d680 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.544024 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb92cab-5996-44c5-97ef-633448d5d680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.544410 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb92cab-5996-44c5-97ef-633448d5d680 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.544634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8d75a-5bbb-44af-a0ed-7746ee76181e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.545301 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb92cab-5996-44c5-97ef-633448d5d680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.545430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8d75a-5bbb-44af-a0ed-7746ee76181e admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.545649 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8d75a-5bbb-44af-a0ed-7746ee76181e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.546033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8d75a-5bbb-44af-a0ed-7746ee76181e admin admin] PATH_INFO: `/v3/projects/5e111ec2c8324461ad7741856f130b9f/users/7406519ab8f843078f40a7da578b4390/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.547405 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb92cab-5996-44c5-97ef-633448d5d680 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.548070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8d75a-5bbb-44af-a0ed-7746ee76181e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e111ec2c8324461ad7741856f130b9f, user_id=7406519ab8f843078f40a7da578b4390, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.551211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8d75a-5bbb-44af-a0ed-7746ee76181e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.575168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 818/1625] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:42 2022] PUT /identity/v3/projects/5e111ec2c8324461ad7741856f130b9f/users/7406519ab8f843078f40a7da578b4390/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:42.577254 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25718ad-516d-4235-9360-4431a49e073a None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.580442 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25718ad-516d-4235-9360-4431a49e073a None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.580628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 808/1626] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:42.580732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25718ad-516d-4235-9360-4431a49e073a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.581829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb92cab-5996-44c5-97ef-633448d5d680 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.597049 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb92cab-5996-44c5-97ef-633448d5d680 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.644638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-f25718ad-516d-4235-9360-4431a49e073a None None] MFA Rules not processed for user `7406519ab8f843078f40a7da578b4390`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:42.646659 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f25718ad-516d-4235-9360-4431a49e073a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:42.728550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515ccf6-0951-4280-b33c-12688d4519e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.729280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515ccf6-0951-4280-b33c-12688d4519e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.729629 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515ccf6-0951-4280-b33c-12688d4519e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.729891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515ccf6-0951-4280-b33c-12688d4519e9 admin admin] PATH_INFO: `/v3/projects/35992455218849b0aef10f30523ab221/users/37fddae702734b48a9a3907fb503e0b6/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.757179 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 819/1627] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:42 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:42.767843 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25718ad-516d-4235-9360-4431a49e073a None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.772028 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515ccf6-0951-4280-b33c-12688d4519e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35992455218849b0aef10f30523ab221, user_id=37fddae702734b48a9a3907fb503e0b6, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.774826 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515ccf6-0951-4280-b33c-12688d4519e9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.816324 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 809/1628] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:42 2022] PUT /identity/v3/projects/35992455218849b0aef10f30523ab221/users/37fddae702734b48a9a3907fb503e0b6/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:23:42.820240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515ccf6-0951-4280-b33c-12688d4519e9 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.835295 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:42.845368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:42.893619 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25718ad-516d-4235-9360-4431a49e073a None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.938314 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515ccf6-0951-4280-b33c-12688d4519e9 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.945825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162583a9-a420-4072-8a3d-69d2cffd2e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.947129 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162583a9-a420-4072-8a3d-69d2cffd2e99 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.947281 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162583a9-a420-4072-8a3d-69d2cffd2e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.947570 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162583a9-a420-4072-8a3d-69d2cffd2e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.948570 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162583a9-a420-4072-8a3d-69d2cffd2e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.950514 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162583a9-a420-4072-8a3d-69d2cffd2e99 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.952252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea0747-b20f-4eed-875f-981be7234f96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.954187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea0747-b20f-4eed-875f-981be7234f96 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.955019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea0747-b20f-4eed-875f-981be7234f96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.955491 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea0747-b20f-4eed-875f-981be7234f96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.966542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 810/1629] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:23:42.969192 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a67081-6084-4b7e-8173-537d242fd94f None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.969547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a67081-6084-4b7e-8173-537d242fd94f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.969909 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a67081-6084-4b7e-8173-537d242fd94f None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.972483 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 811/1630] 10.222.0.48 () {54 vars in 962 bytes} [Mon Aug 1 15:23:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:23:42.973584 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a67081-6084-4b7e-8173-537d242fd94f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.990908 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a67081-6084-4b7e-8173-537d242fd94f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.996693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9166b9-7d4a-4449-b6dd-82de15f0cd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.998100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9166b9-7d4a-4449-b6dd-82de15f0cd44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.998488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9166b9-7d4a-4449-b6dd-82de15f0cd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.998840 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9166b9-7d4a-4449-b6dd-82de15f0cd44 admin admin] PATH_INFO: `/v3/projects/35992455218849b0aef10f30523ab221/users/37fddae702734b48a9a3907fb503e0b6/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:43.001278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9166b9-7d4a-4449-b6dd-82de15f0cd44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35992455218849b0aef10f30523ab221, user_id=37fddae702734b48a9a3907fb503e0b6, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:43.005432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9166b9-7d4a-4449-b6dd-82de15f0cd44 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:43.025703 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 812/1631] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:42 2022] PUT /identity/v3/projects/35992455218849b0aef10f30523ab221/users/37fddae702734b48a9a3907fb503e0b6/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:43.027465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd7d54-40f2-4aba-afe4-fe7a264c6879 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:43.027964 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd7d54-40f2-4aba-afe4-fe7a264c6879 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:43.028295 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd7d54-40f2-4aba-afe4-fe7a264c6879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:43.048381 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea0747-b20f-4eed-875f-981be7234f96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:43.050887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea0747-b20f-4eed-875f-981be7234f96 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:43.063661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 820/1632] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:42 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:43.065834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7af93-236d-4748-a489-e8d50267fdd9 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:43.066140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7af93-236d-4748-a489-e8d50267fdd9 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:43.066422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7af93-236d-4748-a489-e8d50267fdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:43.103169 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-27e7af93-236d-4748-a489-e8d50267fdd9 None None] MFA Rules not processed for user `37fddae702734b48a9a3907fb503e0b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:43.104779 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-27e7af93-236d-4748-a489-e8d50267fdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:43.146345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e2dd7d54-40f2-4aba-afe4-fe7a264c6879 None None] MFA Rules not processed for user `2de0ff99a2b74979a28a8b10e9ebdfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:43.148307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e2dd7d54-40f2-4aba-afe4-fe7a264c6879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:43.189838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 821/1633] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:43 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:43.213138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7af93-236d-4748-a489-e8d50267fdd9 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:43.273901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:43.323846 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7af93-236d-4748-a489-e8d50267fdd9 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:43.332350 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c1b1a-0e6d-42ab-b382-2c76ef4b9ab6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:43.333011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c1b1a-0e6d-42ab-b382-2c76ef4b9ab6 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:43.335176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c1b1a-0e6d-42ab-b382-2c76ef4b9ab6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:43.336088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c1b1a-0e6d-42ab-b382-2c76ef4b9ab6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:43.364151 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c1b1a-0e6d-42ab-b382-2c76ef4b9ab6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:43.365955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c1b1a-0e6d-42ab-b382-2c76ef4b9ab6 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:43.367360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 813/1634] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Aug 1 15:23:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:43.374230 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b386a-baf4-442b-ad2b-bfe756a38899 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:43.375859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b386a-baf4-442b-ad2b-bfe756a38899 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:43.376970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b386a-baf4-442b-ad2b-bfe756a38899 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:43.378962 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 822/1635] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:43 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:43.385019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 814/1636] 10.222.0.48 () {54 vars in 912 bytes} [Mon Aug 1 15:23:43 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:23:43.397632 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02b495-fad4-4f5c-b3af-e506acaa052f None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:43.397932 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02b495-fad4-4f5c-b3af-e506acaa052f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:43.398213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02b495-fad4-4f5c-b3af-e506acaa052f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:43.402884 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b386a-baf4-442b-ad2b-bfe756a38899 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:43.453147 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-af02b495-fad4-4f5c-b3af-e506acaa052f None None] MFA Rules not processed for user `2de0ff99a2b74979a28a8b10e9ebdfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:43.454472 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-af02b495-fad4-4f5c-b3af-e506acaa052f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:43.483785 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 823/1637] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 15:23:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:43.497451 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02b495-fad4-4f5c-b3af-e506acaa052f None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:43.508637 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b386a-baf4-442b-ad2b-bfe756a38899 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:43.518126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02b495-fad4-4f5c-b3af-e506acaa052f None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:43.518869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9e7ea-ccb9-4de6-ad38-bb69e99d3f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:43.519630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9e7ea-ccb9-4de6-ad38-bb69e99d3f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:43.520003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9e7ea-ccb9-4de6-ad38-bb69e99d3f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:43.520605 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9e7ea-ccb9-4de6-ad38-bb69e99d3f90 admin admin] PATH_INFO: `/v3/users/d7efb76dbe374678b5e20f847f1a3dcb` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:43.529378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4d03e-cb41-4a78-9b49-f989ffab5284 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:43.530429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4d03e-cb41-4a78-9b49-f989ffab5284 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:43.531547 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4d03e-cb41-4a78-9b49-f989ffab5284 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:43.533454 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9e7ea-ccb9-4de6-ad38-bb69e99d3f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7efb76dbe374678b5e20f847f1a3dcb)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:43.536143 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9e7ea-ccb9-4de6-ad38-bb69e99d3f90 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:43.536711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4d03e-cb41-4a78-9b49-f989ffab5284 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:43.561050 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:43.561846 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4d03e-cb41-4a78-9b49-f989ffab5284 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:43.568211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4d03e-cb41-4a78-9b49-f989ffab5284 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:43.617364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 824/1638] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:43.653174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cfd9e7ea-ccb9-4de6-ad38-bb69e99d3f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7efb76dbe374678b5e20f847f1a3dcb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:43.653995 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cfd9e7ea-ccb9-4de6-ad38-bb69e99d3f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7efb76dbe374678b5e20f847f1a3dcb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:43.680796 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cfd9e7ea-ccb9-4de6-ad38-bb69e99d3f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7efb76dbe374678b5e20f847f1a3dcb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:43.694154 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cfd9e7ea-ccb9-4de6-ad38-bb69e99d3f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7efb76dbe374678b5e20f847f1a3dcb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:43.715968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cfd9e7ea-ccb9-4de6-ad38-bb69e99d3f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7efb76dbe374678b5e20f847f1a3dcb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:43.747474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 815/1639] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:23:43 2022] DELETE /identity/v3/users/d7efb76dbe374678b5e20f847f1a3dcb => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:43.753429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c49743-dd71-4b51-beff-4cdd983d7084 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:43.753843 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c49743-dd71-4b51-beff-4cdd983d7084 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:43.754242 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c49743-dd71-4b51-beff-4cdd983d7084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:43.799276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-66c49743-dd71-4b51-beff-4cdd983d7084 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:43.800850 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-66c49743-dd71-4b51-beff-4cdd983d7084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:44.008011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 825/1640] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:44.027660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9e7ea-ccb9-4de6-ad38-bb69e99d3f90 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:44.029997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cfd9e7ea-ccb9-4de6-ad38-bb69e99d3f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:44.109644 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9e7ea-ccb9-4de6-ad38-bb69e99d3f90 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:44.116565 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd492ff-0b25-4550-9934-4fb8e0b1a0da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:44.117269 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd492ff-0b25-4550-9934-4fb8e0b1a0da service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:44.117544 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd492ff-0b25-4550-9934-4fb8e0b1a0da service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:44.117915 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd492ff-0b25-4550-9934-4fb8e0b1a0da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:44.138755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd492ff-0b25-4550-9934-4fb8e0b1a0da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:44.141053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd492ff-0b25-4550-9934-4fb8e0b1a0da service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:44.156498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 816/1641] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:44.314057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c49743-dd71-4b51-beff-4cdd983d7084 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:44.316534 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-66c49743-dd71-4b51-beff-4cdd983d7084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:44.334549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c49743-dd71-4b51-beff-4cdd983d7084 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:44.344686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29068d3-b3fe-4d58-bcf2-3dcc6283afcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:44.358730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29068d3-b3fe-4d58-bcf2-3dcc6283afcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:44.359020 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29068d3-b3fe-4d58-bcf2-3dcc6283afcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:44.359302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29068d3-b3fe-4d58-bcf2-3dcc6283afcd admin admin] PATH_INFO: `/v3/users/837717dfd6674ca9ba8da95fe7df3eb4` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:44.361031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29068d3-b3fe-4d58-bcf2-3dcc6283afcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=837717dfd6674ca9ba8da95fe7df3eb4)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:44.362188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: WARNING py.warnings [None req-d29068d3-b3fe-4d58-bcf2-3dcc6283afcd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:23:44.362188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: warnings.warn(msg) Aug 01 15:23:44.362188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: Aug 01 15:23:44.363510 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29068d3-b3fe-4d58-bcf2-3dcc6283afcd admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:44.516204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d29068d3-b3fe-4d58-bcf2-3dcc6283afcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '837717dfd6674ca9ba8da95fe7df3eb4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:44.543722 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d29068d3-b3fe-4d58-bcf2-3dcc6283afcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '837717dfd6674ca9ba8da95fe7df3eb4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:44.544251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d29068d3-b3fe-4d58-bcf2-3dcc6283afcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '837717dfd6674ca9ba8da95fe7df3eb4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:44.568010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d29068d3-b3fe-4d58-bcf2-3dcc6283afcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '837717dfd6674ca9ba8da95fe7df3eb4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:44.594316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d29068d3-b3fe-4d58-bcf2-3dcc6283afcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '837717dfd6674ca9ba8da95fe7df3eb4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:44.605909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 826/1642] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:23:44 2022] DELETE /identity/v3/users/837717dfd6674ca9ba8da95fe7df3eb4 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:44.607537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a07a8f-21e1-4e3e-be64-f83478ea93cd None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:44.607851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a07a8f-21e1-4e3e-be64-f83478ea93cd None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:44.608152 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a07a8f-21e1-4e3e-be64-f83478ea93cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:44.648176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-94a07a8f-21e1-4e3e-be64-f83478ea93cd None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:44.649774 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-94a07a8f-21e1-4e3e-be64-f83478ea93cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:44.746604 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 817/1643] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:44.800433 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29068d3-b3fe-4d58-bcf2-3dcc6283afcd admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:44.802059 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d29068d3-b3fe-4d58-bcf2-3dcc6283afcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:44.844377 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:44.915665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29068d3-b3fe-4d58-bcf2-3dcc6283afcd admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:44.920313 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c63f0e-3af9-4ba8-aa3d-bac2c8d2cd7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:44.921206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c63f0e-3af9-4ba8-aa3d-bac2c8d2cd7a service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:44.921587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c63f0e-3af9-4ba8-aa3d-bac2c8d2cd7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:44.921992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c63f0e-3af9-4ba8-aa3d-bac2c8d2cd7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:44.932933 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c63f0e-3af9-4ba8-aa3d-bac2c8d2cd7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:44.934622 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c63f0e-3af9-4ba8-aa3d-bac2c8d2cd7a service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:44.951671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 827/1644] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:45.524259 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a07a8f-21e1-4e3e-be64-f83478ea93cd None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:45.529438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-94a07a8f-21e1-4e3e-be64-f83478ea93cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:45.555192 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a07a8f-21e1-4e3e-be64-f83478ea93cd None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:45.587124 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c9d89-c548-4fde-81a0-394b3c4c8b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:45.588150 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c9d89-c548-4fde-81a0-394b3c4c8b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:45.592717 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c9d89-c548-4fde-81a0-394b3c4c8b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:45.593148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c9d89-c548-4fde-81a0-394b3c4c8b0a admin admin] PATH_INFO: `/v3/projects/2632844dad0b49f4beb8728d99e19e2a` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:45.597261 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c9d89-c548-4fde-81a0-394b3c4c8b0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2632844dad0b49f4beb8728d99e19e2a)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:45.602130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: WARNING py.warnings [None req-b93c9d89-c548-4fde-81a0-394b3c4c8b0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:23:45.602130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: warnings.warn(msg) Aug 01 15:23:45.602130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: Aug 01 15:23:45.606374 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c9d89-c548-4fde-81a0-394b3c4c8b0a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:45.698257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:45.757944 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b93c9d89-c548-4fde-81a0-394b3c4c8b0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2632844dad0b49f4beb8728d99e19e2a', 'request_id': 'req-b93c9d89-c548-4fde-81a0-394b3c4c8b0a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:45.792737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b93c9d89-c548-4fde-81a0-394b3c4c8b0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2632844dad0b49f4beb8728d99e19e2a', 'request_id': 'req-b93c9d89-c548-4fde-81a0-394b3c4c8b0a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:45.814305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b93c9d89-c548-4fde-81a0-394b3c4c8b0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2632844dad0b49f4beb8728d99e19e2a', 'request_id': 'req-b93c9d89-c548-4fde-81a0-394b3c4c8b0a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:45.851488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b93c9d89-c548-4fde-81a0-394b3c4c8b0a admin admin] The token cache is being invalidate because project 2632844dad0b49f4beb8728d99e19e2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:23:45.851749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b93c9d89-c548-4fde-81a0-394b3c4c8b0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:45.852819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b93c9d89-c548-4fde-81a0-394b3c4c8b0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:45.854545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 818/1645] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:23:45 2022] DELETE /identity/v3/projects/2632844dad0b49f4beb8728d99e19e2a => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:45.858801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c63f0e-3af9-4ba8-aa3d-bac2c8d2cd7a service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:45.882234 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c63f0e-3af9-4ba8-aa3d-bac2c8d2cd7a service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:45.927501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3daaf9-83f5-4a90-b415-3438c09a6ac7 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:45.930122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3daaf9-83f5-4a90-b415-3438c09a6ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:45.930122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3daaf9-83f5-4a90-b415-3438c09a6ac7 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:45.931559 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 819/1646] 10.222.0.48 () {54 vars in 962 bytes} [Mon Aug 1 15:23:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:23:45.943886 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900067ad-296d-45cc-aa99-10cd465fe7ff None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:45.943886 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900067ad-296d-45cc-aa99-10cd465fe7ff None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:45.944333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900067ad-296d-45cc-aa99-10cd465fe7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:45.992192 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-900067ad-296d-45cc-aa99-10cd465fe7ff None None] MFA Rules not processed for user `2de0ff99a2b74979a28a8b10e9ebdfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:46.001668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-900067ad-296d-45cc-aa99-10cd465fe7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:46.005069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c473855b-8ee4-44d3-98ed-7a2695ae4ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:46.005902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473855b-8ee4-44d3-98ed-7a2695ae4ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:46.006440 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473855b-8ee4-44d3-98ed-7a2695ae4ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:46.006960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c473855b-8ee4-44d3-98ed-7a2695ae4ba8 admin admin] PATH_INFO: `/v3/users/7406519ab8f843078f40a7da578b4390` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:46.008488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c473855b-8ee4-44d3-98ed-7a2695ae4ba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7406519ab8f843078f40a7da578b4390)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:46.011005 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c473855b-8ee4-44d3-98ed-7a2695ae4ba8 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:46.100823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c473855b-8ee4-44d3-98ed-7a2695ae4ba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7406519ab8f843078f40a7da578b4390'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:46.113543 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c473855b-8ee4-44d3-98ed-7a2695ae4ba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7406519ab8f843078f40a7da578b4390'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:46.114858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c473855b-8ee4-44d3-98ed-7a2695ae4ba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7406519ab8f843078f40a7da578b4390'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:46.135785 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c473855b-8ee4-44d3-98ed-7a2695ae4ba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7406519ab8f843078f40a7da578b4390'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:46.154690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c473855b-8ee4-44d3-98ed-7a2695ae4ba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7406519ab8f843078f40a7da578b4390'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:46.158527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 820/1647] 10.222.0.48 () {58 vars in 1059 bytes} [Mon Aug 1 15:23:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:23:46.162312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 828/1648] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:23:45 2022] DELETE /identity/v3/users/7406519ab8f843078f40a7da578b4390 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:46.171514 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd39ff1-6c05-4164-b2e2-a99dfe90d68f None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:46.172841 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd39ff1-6c05-4164-b2e2-a99dfe90d68f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:46.173297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd39ff1-6c05-4164-b2e2-a99dfe90d68f None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:46.179170 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 821/1649] 10.222.0.48 () {54 vars in 912 bytes} [Mon Aug 1 15:23:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:23:46.182432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7867c43-6c67-4190-8437-bc11a34129b7 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:46.182729 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7867c43-6c67-4190-8437-bc11a34129b7 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:46.183007 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7867c43-6c67-4190-8437-bc11a34129b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:46.230950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-e7867c43-6c67-4190-8437-bc11a34129b7 None None] MFA Rules not processed for user `2de0ff99a2b74979a28a8b10e9ebdfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:46.232422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e7867c43-6c67-4190-8437-bc11a34129b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:46.276894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:46.366813 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 829/1650] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 15:23:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:23:46.373701 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd39ff1-6c05-4164-b2e2-a99dfe90d68f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:46.376074 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6dd39ff1-6c05-4164-b2e2-a99dfe90d68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:46.502105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd39ff1-6c05-4164-b2e2-a99dfe90d68f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:46.502105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b579817-6232-42fc-8bd6-82d71cd6b79c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:46.502105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b579817-6232-42fc-8bd6-82d71cd6b79c service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:46.502105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b579817-6232-42fc-8bd6-82d71cd6b79c service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:46.503100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b579817-6232-42fc-8bd6-82d71cd6b79c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:46.527098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b579817-6232-42fc-8bd6-82d71cd6b79c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:46.534364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b579817-6232-42fc-8bd6-82d71cd6b79c service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:46.565717 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 822/1651] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:23:47.364585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7867c43-6c67-4190-8437-bc11a34129b7 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:47.365066 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e7867c43-6c67-4190-8437-bc11a34129b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:47.653529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7867c43-6c67-4190-8437-bc11a34129b7 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:47.666914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:47.668193 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:47.668675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:47.669132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b admin admin] PATH_INFO: `/v3/projects/b2de96a47291485e983fd527a03b2e27` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:47.671435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2de96a47291485e983fd527a03b2e27)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:47.686335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:47.810393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2de96a47291485e983fd527a03b2e27', 'request_id': 'req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:47.840686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2de96a47291485e983fd527a03b2e27', 'request_id': 'req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:47.849597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2de96a47291485e983fd527a03b2e27', 'request_id': 'req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:47.912866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b admin admin] The token cache is being invalidate because project b2de96a47291485e983fd527a03b2e27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:23:47.912866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:47.913752 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:47.919005 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 830/1652] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:23:47 2022] DELETE /identity/v3/projects/b2de96a47291485e983fd527a03b2e27 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:47.920363 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b579817-6232-42fc-8bd6-82d71cd6b79c service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:47.937274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b579817-6232-42fc-8bd6-82d71cd6b79c service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:48.035773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:48.057837 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf38e5f-fa22-4f6c-8870-991701779c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:48.059498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf38e5f-fa22-4f6c-8870-991701779c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:48.062031 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf38e5f-fa22-4f6c-8870-991701779c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:48.062407 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf38e5f-fa22-4f6c-8870-991701779c86 admin admin] PATH_INFO: `/v3/users/37fddae702734b48a9a3907fb503e0b6` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:48.067661 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf38e5f-fa22-4f6c-8870-991701779c86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37fddae702734b48a9a3907fb503e0b6)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:48.073418 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf38e5f-fa22-4f6c-8870-991701779c86 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:48.189897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-dcf38e5f-fa22-4f6c-8870-991701779c86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37fddae702734b48a9a3907fb503e0b6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:48.191676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-dcf38e5f-fa22-4f6c-8870-991701779c86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37fddae702734b48a9a3907fb503e0b6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:48.240894 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-dcf38e5f-fa22-4f6c-8870-991701779c86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37fddae702734b48a9a3907fb503e0b6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:48.278254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-dcf38e5f-fa22-4f6c-8870-991701779c86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37fddae702734b48a9a3907fb503e0b6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:48.292918 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-dcf38e5f-fa22-4f6c-8870-991701779c86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37fddae702734b48a9a3907fb503e0b6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:48.308044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 823/1653] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:23:47 2022] DELETE /identity/v3/users/37fddae702734b48a9a3907fb503e0b6 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:49.823821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:49.825452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:49.875863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c60f2-60a7-4c5d-8dbd-32c3eb77c53b admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:50.138025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf38e5f-fa22-4f6c-8870-991701779c86 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:50.142329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-dcf38e5f-fa22-4f6c-8870-991701779c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:50.196066 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955c177-37a4-49cd-8c54-b2182e0572db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:50.203335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955c177-37a4-49cd-8c54-b2182e0572db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:50.203699 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955c177-37a4-49cd-8c54-b2182e0572db admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:50.203988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955c177-37a4-49cd-8c54-b2182e0572db admin admin] PATH_INFO: `/v3/projects/5e111ec2c8324461ad7741856f130b9f` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:50.208901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955c177-37a4-49cd-8c54-b2182e0572db admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e111ec2c8324461ad7741856f130b9f)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:50.211953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955c177-37a4-49cd-8c54-b2182e0572db admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:50.227086 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:50.376127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf38e5f-fa22-4f6c-8870-991701779c86 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:50.377604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0955c177-37a4-49cd-8c54-b2182e0572db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e111ec2c8324461ad7741856f130b9f', 'request_id': 'req-0955c177-37a4-49cd-8c54-b2182e0572db'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:50.393086 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0955c177-37a4-49cd-8c54-b2182e0572db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e111ec2c8324461ad7741856f130b9f', 'request_id': 'req-0955c177-37a4-49cd-8c54-b2182e0572db'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:50.416342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0955c177-37a4-49cd-8c54-b2182e0572db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e111ec2c8324461ad7741856f130b9f', 'request_id': 'req-0955c177-37a4-49cd-8c54-b2182e0572db'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:50.447974 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0955c177-37a4-49cd-8c54-b2182e0572db admin admin] The token cache is being invalidate because project 5e111ec2c8324461ad7741856f130b9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:23:50.448278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0955c177-37a4-49cd-8c54-b2182e0572db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:50.449151 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0955c177-37a4-49cd-8c54-b2182e0572db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:50.450811 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 831/1654] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:23:49 2022] DELETE /identity/v3/projects/5e111ec2c8324461ad7741856f130b9f => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:50.458866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686954e-7427-40f2-ad4c-c5d699a6a383 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:50.459302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686954e-7427-40f2-ad4c-c5d699a6a383 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:50.459589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686954e-7427-40f2-ad4c-c5d699a6a383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:50.503930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-5686954e-7427-40f2-ad4c-c5d699a6a383 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:50.505249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5686954e-7427-40f2-ad4c-c5d699a6a383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:50.545926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cab18e-4736-4972-aa51-2e8c62c0f682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:50.546652 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cab18e-4736-4972-aa51-2e8c62c0f682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:50.546934 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cab18e-4736-4972-aa51-2e8c62c0f682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:50.547279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cab18e-4736-4972-aa51-2e8c62c0f682 admin admin] PATH_INFO: `/v3/projects/35992455218849b0aef10f30523ab221` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:50.549540 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cab18e-4736-4972-aa51-2e8c62c0f682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35992455218849b0aef10f30523ab221)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:50.559019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cab18e-4736-4972-aa51-2e8c62c0f682 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:50.644045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:50.731194 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82cab18e-4736-4972-aa51-2e8c62c0f682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35992455218849b0aef10f30523ab221', 'request_id': 'req-82cab18e-4736-4972-aa51-2e8c62c0f682'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:50.753950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 832/1655] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:50.756546 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bdc1d-ed14-4c14-a925-8302fb7a5b1a None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:50.756546 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bdc1d-ed14-4c14-a925-8302fb7a5b1a None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:50.756546 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bdc1d-ed14-4c14-a925-8302fb7a5b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:50.769457 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82cab18e-4736-4972-aa51-2e8c62c0f682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35992455218849b0aef10f30523ab221', 'request_id': 'req-82cab18e-4736-4972-aa51-2e8c62c0f682'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:50.796278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82cab18e-4736-4972-aa51-2e8c62c0f682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35992455218849b0aef10f30523ab221', 'request_id': 'req-82cab18e-4736-4972-aa51-2e8c62c0f682'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:50.819503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82cab18e-4736-4972-aa51-2e8c62c0f682 admin admin] The token cache is being invalidate because project 35992455218849b0aef10f30523ab221 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:23:50.819801 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82cab18e-4736-4972-aa51-2e8c62c0f682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:50.820682 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82cab18e-4736-4972-aa51-2e8c62c0f682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:50.830951 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 824/1656] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:23:50 2022] DELETE /identity/v3/projects/35992455218849b0aef10f30523ab221 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:50.843969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-e67bdc1d-ed14-4c14-a925-8302fb7a5b1a None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:50.845690 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90874ca7-98f5-4a89-b302-c0bc7c1df2d4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:50.846006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90874ca7-98f5-4a89-b302-c0bc7c1df2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:50.846286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90874ca7-98f5-4a89-b302-c0bc7c1df2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:50.848448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e67bdc1d-ed14-4c14-a925-8302fb7a5b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:50.921230 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-90874ca7-98f5-4a89-b302-c0bc7c1df2d4 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:50.924213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-90874ca7-98f5-4a89-b302-c0bc7c1df2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:51.090458 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 833/1657] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:51.095153 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bdc1d-ed14-4c14-a925-8302fb7a5b1a None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:51.152595 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bdc1d-ed14-4c14-a925-8302fb7a5b1a None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:51.180642 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973dcf60-af74-4174-b270-109d09b21b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:51.181383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dcf60-af74-4174-b270-109d09b21b8a admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:51.181701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dcf60-af74-4174-b270-109d09b21b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:51.183056 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973dcf60-af74-4174-b270-109d09b21b8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:51.191521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973dcf60-af74-4174-b270-109d09b21b8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:51.191521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973dcf60-af74-4174-b270-109d09b21b8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:51.191521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973dcf60-af74-4174-b270-109d09b21b8a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:51.197989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 825/1658] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:51.200396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c847ad-9227-4d70-ac0e-7ec922e178bd None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:51.200823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c847ad-9227-4d70-ac0e-7ec922e178bd None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:51.201234 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c847ad-9227-4d70-ac0e-7ec922e178bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:51.256768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 834/1659] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:51.257918 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973dcf60-af74-4174-b270-109d09b21b8a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:51.267200 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a1c847ad-9227-4d70-ac0e-7ec922e178bd None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:51.269325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a1c847ad-9227-4d70-ac0e-7ec922e178bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:51.275150 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973dcf60-af74-4174-b270-109d09b21b8a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:51.287752 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad796c-d5dc-4673-b2d6-c21ade6fb429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:51.288750 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad796c-d5dc-4673-b2d6-c21ade6fb429 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:51.289086 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad796c-d5dc-4673-b2d6-c21ade6fb429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:51.289372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad796c-d5dc-4673-b2d6-c21ade6fb429 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:51.291001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad796c-d5dc-4673-b2d6-c21ade6fb429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:51.292361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 826/1660] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:51.297650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c847ad-9227-4d70-ac0e-7ec922e178bd None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:51.299308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad796c-d5dc-4673-b2d6-c21ade6fb429 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:51.340809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c847ad-9227-4d70-ac0e-7ec922e178bd None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:51.352555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:51.356188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807f3d4-0086-4822-b691-379f9a387f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:51.356993 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807f3d4-0086-4822-b691-379f9a387f39 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:51.358880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807f3d4-0086-4822-b691-379f9a387f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:51.362841 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 835/1661] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:23:51 2022] POST /identity/v3/projects => generated 355 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:51.367829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad796c-d5dc-4673-b2d6-c21ade6fb429 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:51.373013 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807f3d4-0086-4822-b691-379f9a387f39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:51.376902 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807f3d4-0086-4822-b691-379f9a387f39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:51.378567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807f3d4-0086-4822-b691-379f9a387f39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:51.380207 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807f3d4-0086-4822-b691-379f9a387f39 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:51.419695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 827/1662] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:51.423250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807f3d4-0086-4822-b691-379f9a387f39 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:51.448601 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807f3d4-0086-4822-b691-379f9a387f39 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:51.512530 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad796c-d5dc-4673-b2d6-c21ade6fb429 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:51.523997 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89857bba-97fa-4bd6-bce1-66c51e997922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:51.524699 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89857bba-97fa-4bd6-bce1-66c51e997922 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:51.524999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89857bba-97fa-4bd6-bce1-66c51e997922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:51.525276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89857bba-97fa-4bd6-bce1-66c51e997922 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:51.526451 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89857bba-97fa-4bd6-bce1-66c51e997922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:51.530576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89857bba-97fa-4bd6-bce1-66c51e997922 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:51.578168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 836/1663] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:23:51 2022] POST /identity/v3/users => generated 278 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:51.582271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89857bba-97fa-4bd6-bce1-66c51e997922 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:51.609799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89857bba-97fa-4bd6-bce1-66c51e997922 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:51.629283 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3573c-3b54-44d0-9afb-fadae6e0a5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:51.629975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3573c-3b54-44d0-9afb-fadae6e0a5af admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:51.630965 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3573c-3b54-44d0-9afb-fadae6e0a5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:51.630965 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3573c-3b54-44d0-9afb-fadae6e0a5af admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:51.631214 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3573c-3b54-44d0-9afb-fadae6e0a5af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:51.639970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd06de5-c382-4fa9-91a9-6bf579fcdd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:51.639970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd06de5-c382-4fa9-91a9-6bf579fcdd05 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:51.639970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd06de5-c382-4fa9-91a9-6bf579fcdd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:51.639970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd06de5-c382-4fa9-91a9-6bf579fcdd05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:51.640549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd06de5-c382-4fa9-91a9-6bf579fcdd05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:51.641730 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd06de5-c382-4fa9-91a9-6bf579fcdd05 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:51.645810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3573c-3b54-44d0-9afb-fadae6e0a5af admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:51.679151 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 837/1664] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:51.683206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3573c-3b54-44d0-9afb-fadae6e0a5af admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:51.705898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3573c-3b54-44d0-9afb-fadae6e0a5af admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:51.720792 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d87597-ac39-472f-a53b-7849967779f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:51.725196 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d87597-ac39-472f-a53b-7849967779f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:51.725515 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d87597-ac39-472f-a53b-7849967779f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:51.725939 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d87597-ac39-472f-a53b-7849967779f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:51.733193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 828/1665] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:23:51 2022] POST /identity/v3/projects => generated 383 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:51.737091 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd06de5-c382-4fa9-91a9-6bf579fcdd05 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:51.742747 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d87597-ac39-472f-a53b-7849967779f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:51.745065 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d87597-ac39-472f-a53b-7849967779f4 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:51.772673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 838/1666] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:51.776028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d87597-ac39-472f-a53b-7849967779f4 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:51.812651 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:51.815927 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d87597-ac39-472f-a53b-7849967779f4 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:51.975191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd06de5-c382-4fa9-91a9-6bf579fcdd05 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:51.985182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0f8c0-9979-4b57-9fec-f5d9af70218e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:51.987582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0f8c0-9979-4b57-9fec-f5d9af70218e admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:51.988075 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0f8c0-9979-4b57-9fec-f5d9af70218e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:51.988209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ff499-a9e7-4f24-bc48-49311ed4c6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:51.988567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0f8c0-9979-4b57-9fec-f5d9af70218e admin admin] PATH_INFO: `/v3/projects/49cbcc6897e2410588125aba7c6fbb85/users/3a5c31b36e5249e890db4ed180faaa71/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:51.989286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ff499-a9e7-4f24-bc48-49311ed4c6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:51.989700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ff499-a9e7-4f24-bc48-49311ed4c6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:51.990209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ff499-a9e7-4f24-bc48-49311ed4c6b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:51.991669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ff499-a9e7-4f24-bc48-49311ed4c6b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:51.995967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ff499-a9e7-4f24-bc48-49311ed4c6b0 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:52.056516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0f8c0-9979-4b57-9fec-f5d9af70218e admin admin] RBAC: Authorizing `identity:create_grant(project_id=49cbcc6897e2410588125aba7c6fbb85, user_id=3a5c31b36e5249e890db4ed180faaa71, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:52.060284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 829/1667] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:23:51 2022] POST /identity/v3/users => generated 292 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:52.061302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ff499-a9e7-4f24-bc48-49311ed4c6b0 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:52.069346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0f8c0-9979-4b57-9fec-f5d9af70218e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:52.082874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ff499-a9e7-4f24-bc48-49311ed4c6b0 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:52.094827 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5debcdf7-01f1-4c23-ab4f-60028530a2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:52.095538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debcdf7-01f1-4c23-ab4f-60028530a2c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:52.095820 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debcdf7-01f1-4c23-ab4f-60028530a2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:52.096642 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5debcdf7-01f1-4c23-ab4f-60028530a2c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:52.096995 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debcdf7-01f1-4c23-ab4f-60028530a2c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:52.098754 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5debcdf7-01f1-4c23-ab4f-60028530a2c9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:52.123885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 839/1668] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:51 2022] PUT /identity/v3/projects/49cbcc6897e2410588125aba7c6fbb85/users/3a5c31b36e5249e890db4ed180faaa71/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:52.128339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 830/1669] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:52.129954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0f8c0-9979-4b57-9fec-f5d9af70218e admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:52.135870 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df23a19-8b93-4d20-bc54-2a699d4a6fa6 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:52.136255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df23a19-8b93-4d20-bc54-2a699d4a6fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:52.136417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df23a19-8b93-4d20-bc54-2a699d4a6fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:52.193344 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6df23a19-8b93-4d20-bc54-2a699d4a6fa6 None None] MFA Rules not processed for user `3a5c31b36e5249e890db4ed180faaa71`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:52.195638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6df23a19-8b93-4d20-bc54-2a699d4a6fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:52.346844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 831/1670] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:52 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:23:52.348139 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df23a19-8b93-4d20-bc54-2a699d4a6fa6 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:52.354027 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:52.408372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0f8c0-9979-4b57-9fec-f5d9af70218e admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:52.430001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4b2a9-bbda-4fab-9adb-b3b603e530de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:52.431097 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4b2a9-bbda-4fab-9adb-b3b603e530de admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:52.431586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4b2a9-bbda-4fab-9adb-b3b603e530de admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:52.432071 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4b2a9-bbda-4fab-9adb-b3b603e530de admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:52.433388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4b2a9-bbda-4fab-9adb-b3b603e530de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:52.436502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4b2a9-bbda-4fab-9adb-b3b603e530de admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:52.498508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 840/1671] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:52.500086 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4b2a9-bbda-4fab-9adb-b3b603e530de admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:52.530025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4b2a9-bbda-4fab-9adb-b3b603e530de admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:52.551538 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6d83a-51a7-43cd-a916-b9fcfb7a8a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:52.552827 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6d83a-51a7-43cd-a916-b9fcfb7a8a1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:52.553300 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6d83a-51a7-43cd-a916-b9fcfb7a8a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:52.553803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6d83a-51a7-43cd-a916-b9fcfb7a8a1e admin admin] PATH_INFO: `/v3/projects/85e24ab0539d4acc8447b8b898124707/users/b538ba9d25bb4d669c6b5255ad0528aa/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:52.565137 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df23a19-8b93-4d20-bc54-2a699d4a6fa6 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:52.577085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc242e3-fb8a-4a79-8f11-a0be764f2576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:52.578530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc242e3-fb8a-4a79-8f11-a0be764f2576 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:52.578844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc242e3-fb8a-4a79-8f11-a0be764f2576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:52.579141 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc242e3-fb8a-4a79-8f11-a0be764f2576 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:52.579840 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc242e3-fb8a-4a79-8f11-a0be764f2576 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:52.585737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc242e3-fb8a-4a79-8f11-a0be764f2576 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:52.633617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6d83a-51a7-43cd-a916-b9fcfb7a8a1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=85e24ab0539d4acc8447b8b898124707, user_id=b538ba9d25bb4d669c6b5255ad0528aa, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:52.648782 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 832/1672] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:23:52 2022] POST /identity/v3/projects => generated 355 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:52.653107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc242e3-fb8a-4a79-8f11-a0be764f2576 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:52.656673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6d83a-51a7-43cd-a916-b9fcfb7a8a1e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:52.670248 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:52.705798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 841/1673] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:52 2022] PUT /identity/v3/projects/85e24ab0539d4acc8447b8b898124707/users/b538ba9d25bb4d669c6b5255ad0528aa/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:52.718790 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52009287-0ecf-4310-aee2-f0ebace0b7a3 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:52.719083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52009287-0ecf-4310-aee2-f0ebace0b7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:52.719414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52009287-0ecf-4310-aee2-f0ebace0b7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:52.795716 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-52009287-0ecf-4310-aee2-f0ebace0b7a3 None None] MFA Rules not processed for user `b538ba9d25bb4d669c6b5255ad0528aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:52.802439 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-52009287-0ecf-4310-aee2-f0ebace0b7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:52.852527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc242e3-fb8a-4a79-8f11-a0be764f2576 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:52.974980 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 842/1674] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:52.976250 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52009287-0ecf-4310-aee2-f0ebace0b7a3 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:53.007617 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42537a2-82fe-43f1-9aa9-68e1abbd922b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:53.008800 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42537a2-82fe-43f1-9aa9-68e1abbd922b admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:53.009342 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42537a2-82fe-43f1-9aa9-68e1abbd922b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:53.009867 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42537a2-82fe-43f1-9aa9-68e1abbd922b admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:53.015927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42537a2-82fe-43f1-9aa9-68e1abbd922b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:53.022564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42537a2-82fe-43f1-9aa9-68e1abbd922b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:53.097747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 833/1675] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:23:52 2022] POST /identity/v3/users => generated 284 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:53.102870 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42537a2-82fe-43f1-9aa9-68e1abbd922b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:53.124553 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42537a2-82fe-43f1-9aa9-68e1abbd922b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:53.128594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52009287-0ecf-4310-aee2-f0ebace0b7a3 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:53.138712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc802fc-4488-4713-9f24-76281d82f7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:53.140838 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc802fc-4488-4713-9f24-76281d82f7f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:53.143206 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc802fc-4488-4713-9f24-76281d82f7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:53.143737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc802fc-4488-4713-9f24-76281d82f7f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:53.144774 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc802fc-4488-4713-9f24-76281d82f7f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:53.149018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01998656-9df3-43ad-aca9-b6984530b6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:53.149730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01998656-9df3-43ad-aca9-b6984530b6da admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:53.150040 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01998656-9df3-43ad-aca9-b6984530b6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:53.150315 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01998656-9df3-43ad-aca9-b6984530b6da admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:53.151120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01998656-9df3-43ad-aca9-b6984530b6da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:53.152191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc802fc-4488-4713-9f24-76281d82f7f1 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:53.157149 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01998656-9df3-43ad-aca9-b6984530b6da admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:53.192678 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 834/1676] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:23:53.206428 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc802fc-4488-4713-9f24-76281d82f7f1 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:53.230127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc802fc-4488-4713-9f24-76281d82f7f1 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:53.245688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 843/1677] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:23:52 2022] POST /identity/v3/projects => generated 383 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:53.246808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01998656-9df3-43ad-aca9-b6984530b6da admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:53.328440 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:53.382448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01998656-9df3-43ad-aca9-b6984530b6da admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:53.398360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b639929-f490-4561-9abd-c44526c0e177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:53.399020 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b639929-f490-4561-9abd-c44526c0e177 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:53.399325 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b639929-f490-4561-9abd-c44526c0e177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:53.399579 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b639929-f490-4561-9abd-c44526c0e177 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:53.407119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b639929-f490-4561-9abd-c44526c0e177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:53.409600 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b639929-f490-4561-9abd-c44526c0e177 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:53.436443 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f67bf-23e3-424b-8d00-e03d4cb2be59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:53.437344 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f67bf-23e3-424b-8d00-e03d4cb2be59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:53.437728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f67bf-23e3-424b-8d00-e03d4cb2be59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:53.439220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f67bf-23e3-424b-8d00-e03d4cb2be59 admin admin] PATH_INFO: `/v3/projects/729155ca3de245a08b6934dc7119d334/users/4da5835c823c4568bace5b9d7afd6614/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:53.460379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 844/1678] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:23:53 2022] POST /identity/v3/users => generated 298 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:53.467167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b639929-f490-4561-9abd-c44526c0e177 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:53.476242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f67bf-23e3-424b-8d00-e03d4cb2be59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=729155ca3de245a08b6934dc7119d334, user_id=4da5835c823c4568bace5b9d7afd6614, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:53.479237 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f67bf-23e3-424b-8d00-e03d4cb2be59 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:53.489074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b639929-f490-4561-9abd-c44526c0e177 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:53.507780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 835/1679] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:53 2022] PUT /identity/v3/projects/729155ca3de245a08b6934dc7119d334/users/4da5835c823c4568bace5b9d7afd6614/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:53.508229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c9e95-f1e1-4597-9ce0-2d862c0f7446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:53.508229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c9e95-f1e1-4597-9ce0-2d862c0f7446 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:53.508229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c9e95-f1e1-4597-9ce0-2d862c0f7446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:53.509406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f67bf-23e3-424b-8d00-e03d4cb2be59 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:53.509830 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c9e95-f1e1-4597-9ce0-2d862c0f7446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:53.512279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c9e95-f1e1-4597-9ce0-2d862c0f7446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:53.517767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c9e95-f1e1-4597-9ce0-2d862c0f7446 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:53.550607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 845/1680] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:53.556256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:53.556891 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c9e95-f1e1-4597-9ce0-2d862c0f7446 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:53.583329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c9e95-f1e1-4597-9ce0-2d862c0f7446 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:53.664835 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f67bf-23e3-424b-8d00-e03d4cb2be59 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:53.678554 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e831228-4ff3-4d54-8366-615e7611cbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:53.679378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e831228-4ff3-4d54-8366-615e7611cbfc admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:53.679759 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e831228-4ff3-4d54-8366-615e7611cbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:53.680137 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e831228-4ff3-4d54-8366-615e7611cbfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:53.680940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e831228-4ff3-4d54-8366-615e7611cbfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:53.682840 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e831228-4ff3-4d54-8366-615e7611cbfc admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:53.699322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a3ddd-3017-4d6e-8945-503790a7783d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:53.700118 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a3ddd-3017-4d6e-8945-503790a7783d admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:53.700400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a3ddd-3017-4d6e-8945-503790a7783d admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:53.700676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a3ddd-3017-4d6e-8945-503790a7783d admin admin] PATH_INFO: `/v3/projects/0cb3cb6d940d4dcc9b4a2e8417ecd9a6/users/fc8a9e1c993a4232b220ca70898b826c/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:53.716257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 836/1681] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:53.717586 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e831228-4ff3-4d54-8366-615e7611cbfc admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:53.740095 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e831228-4ff3-4d54-8366-615e7611cbfc admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:53.747841 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb98e1-f5ed-4d60-8244-9ede81cb6fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:53.748691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb98e1-f5ed-4d60-8244-9ede81cb6fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:53.749137 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb98e1-f5ed-4d60-8244-9ede81cb6fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:53.749515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb98e1-f5ed-4d60-8244-9ede81cb6fc0 admin admin] PATH_INFO: `/v3/projects/729155ca3de245a08b6934dc7119d334/users/4da5835c823c4568bace5b9d7afd6614/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:53.753494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb98e1-f5ed-4d60-8244-9ede81cb6fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=729155ca3de245a08b6934dc7119d334, user_id=4da5835c823c4568bace5b9d7afd6614, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:53.759592 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb98e1-f5ed-4d60-8244-9ede81cb6fc0 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:53.782980 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a3ddd-3017-4d6e-8945-503790a7783d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cb3cb6d940d4dcc9b4a2e8417ecd9a6, user_id=fc8a9e1c993a4232b220ca70898b826c, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:53.786608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 837/1682] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:53 2022] PUT /identity/v3/projects/729155ca3de245a08b6934dc7119d334/users/4da5835c823c4568bace5b9d7afd6614/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:53.799231 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a3ddd-3017-4d6e-8945-503790a7783d admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:53.810686 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5fb71-6b2b-411c-9d2a-70a5bf6161e1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:53.811121 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5fb71-6b2b-411c-9d2a-70a5bf6161e1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:53.811518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5fb71-6b2b-411c-9d2a-70a5bf6161e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:53.861584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 846/1683] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:53 2022] PUT /identity/v3/projects/0cb3cb6d940d4dcc9b4a2e8417ecd9a6/users/fc8a9e1c993a4232b220ca70898b826c/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:53.866207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a3ddd-3017-4d6e-8945-503790a7783d admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:53.893420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f8c5fb71-6b2b-411c-9d2a-70a5bf6161e1 None None] MFA Rules not processed for user `4da5835c823c4568bace5b9d7afd6614`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:53.894786 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f8c5fb71-6b2b-411c-9d2a-70a5bf6161e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:54.024046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a3ddd-3017-4d6e-8945-503790a7783d admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:54.037267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9a897-84e8-455a-a03c-cd0fe5d295e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:54.037961 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9a897-84e8-455a-a03c-cd0fe5d295e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:54.038247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9a897-84e8-455a-a03c-cd0fe5d295e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:54.038522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9a897-84e8-455a-a03c-cd0fe5d295e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:54.039244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9a897-84e8-455a-a03c-cd0fe5d295e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:54.043321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9a897-84e8-455a-a03c-cd0fe5d295e4 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:54.062603 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 838/1684] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:53 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:23:54.074031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 847/1685] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:23:54.074154 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5fb71-6b2b-411c-9d2a-70a5bf6161e1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:54.080086 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9a897-84e8-455a-a03c-cd0fe5d295e4 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:54.083703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-00d9a897-84e8-455a-a03c-cd0fe5d295e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:54.091167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5fb71-6b2b-411c-9d2a-70a5bf6161e1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:54.105682 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e95d6-9cd8-45b2-ba20-68ed0b4cded6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:54.106449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e95d6-9cd8-45b2-ba20-68ed0b4cded6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:54.106726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e95d6-9cd8-45b2-ba20-68ed0b4cded6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:54.107003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e95d6-9cd8-45b2-ba20-68ed0b4cded6 admin admin] PATH_INFO: `/v3/projects/0cb3cb6d940d4dcc9b4a2e8417ecd9a6/users/fc8a9e1c993a4232b220ca70898b826c/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:54.115356 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e95d6-9cd8-45b2-ba20-68ed0b4cded6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cb3cb6d940d4dcc9b4a2e8417ecd9a6, user_id=fc8a9e1c993a4232b220ca70898b826c, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:54.127214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e95d6-9cd8-45b2-ba20-68ed0b4cded6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:54.152881 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 839/1686] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:23:54 2022] PUT /identity/v3/projects/0cb3cb6d940d4dcc9b4a2e8417ecd9a6/users/fc8a9e1c993a4232b220ca70898b826c/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:23:54.165168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594f058-63dd-4028-9d6b-13351845c1f3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:54.165614 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594f058-63dd-4028-9d6b-13351845c1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:54.166930 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594f058-63dd-4028-9d6b-13351845c1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:54.205421 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:54.213860 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a594f058-63dd-4028-9d6b-13351845c1f3 None None] MFA Rules not processed for user `fc8a9e1c993a4232b220ca70898b826c`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:54.216111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a594f058-63dd-4028-9d6b-13351845c1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:54.266611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9a897-84e8-455a-a03c-cd0fe5d295e4 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:54.302261 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:54.323319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 840/1687] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:54.331032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594f058-63dd-4028-9d6b-13351845c1f3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:54.332760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a594f058-63dd-4028-9d6b-13351845c1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:54.383942 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a94f33-1352-4e26-85a9-f29f05d8e375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:54.384818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a94f33-1352-4e26-85a9-f29f05d8e375 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:54.385149 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a94f33-1352-4e26-85a9-f29f05d8e375 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:54.385452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a94f33-1352-4e26-85a9-f29f05d8e375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:54.423648 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594f058-63dd-4028-9d6b-13351845c1f3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:54.428644 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54f7c7-ad32-4dfb-9239-7067754e7dea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:54.429631 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54f7c7-ad32-4dfb-9239-7067754e7dea service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:54.430112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54f7c7-ad32-4dfb-9239-7067754e7dea service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:54.430591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54f7c7-ad32-4dfb-9239-7067754e7dea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:54.443510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54f7c7-ad32-4dfb-9239-7067754e7dea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:54.446255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54f7c7-ad32-4dfb-9239-7067754e7dea service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:54.474984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 841/1688] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:54.509037 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a94f33-1352-4e26-85a9-f29f05d8e375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:54.518370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a94f33-1352-4e26-85a9-f29f05d8e375 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:54.547264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 848/1689] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:54 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:23:54.727687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54f7c7-ad32-4dfb-9239-7067754e7dea service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:54.756906 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54f7c7-ad32-4dfb-9239-7067754e7dea service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:54.757509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a94f33-1352-4e26-85a9-f29f05d8e375 service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:54.769094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6518ae9-5fa9-4de3-b405-2ef66f894e82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:54.769383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a94f33-1352-4e26-85a9-f29f05d8e375 service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:54.771441 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6518ae9-5fa9-4de3-b405-2ef66f894e82 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:54.772033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6518ae9-5fa9-4de3-b405-2ef66f894e82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:54.773859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6518ae9-5fa9-4de3-b405-2ef66f894e82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:54.784113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7188d-5fe1-4589-bf2a-b38ad4ef22a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:54.784832 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7188d-5fe1-4589-bf2a-b38ad4ef22a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:54.785119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7188d-5fe1-4589-bf2a-b38ad4ef22a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:54.785394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7188d-5fe1-4589-bf2a-b38ad4ef22a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:54.933183 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7188d-5fe1-4589-bf2a-b38ad4ef22a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:54.944111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7188d-5fe1-4589-bf2a-b38ad4ef22a7 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:54.991668 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 849/1690] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:54 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:23:55.002064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6518ae9-5fa9-4de3-b405-2ef66f894e82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:55.007605 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6518ae9-5fa9-4de3-b405-2ef66f894e82 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:55.029244 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 842/1691] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:55.342554 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7188d-5fe1-4589-bf2a-b38ad4ef22a7 service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:55.344684 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-ecb7188d-5fe1-4589-bf2a-b38ad4ef22a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:55.447618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:55.509437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7188d-5fe1-4589-bf2a-b38ad4ef22a7 service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:55.521903 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085f423-2f5d-44f9-b043-25ad4a4eb10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:55.522757 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085f423-2f5d-44f9-b043-25ad4a4eb10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:55.523162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085f423-2f5d-44f9-b043-25ad4a4eb10e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:55.523603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085f423-2f5d-44f9-b043-25ad4a4eb10e admin admin] PATH_INFO: `/v3/users/5f23c0f21bf54515b8cdc03d415213c5` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:55.525103 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085f423-2f5d-44f9-b043-25ad4a4eb10e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f23c0f21bf54515b8cdc03d415213c5)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:55.527983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085f423-2f5d-44f9-b043-25ad4a4eb10e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:55.626915 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8085f423-2f5d-44f9-b043-25ad4a4eb10e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f23c0f21bf54515b8cdc03d415213c5'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:55.656568 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8085f423-2f5d-44f9-b043-25ad4a4eb10e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f23c0f21bf54515b8cdc03d415213c5'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:55.657292 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8085f423-2f5d-44f9-b043-25ad4a4eb10e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f23c0f21bf54515b8cdc03d415213c5'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:55.672863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8085f423-2f5d-44f9-b043-25ad4a4eb10e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f23c0f21bf54515b8cdc03d415213c5'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:55.712769 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8085f423-2f5d-44f9-b043-25ad4a4eb10e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f23c0f21bf54515b8cdc03d415213c5'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:55.727925 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 850/1692] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:23:55 2022] DELETE /identity/v3/users/5f23c0f21bf54515b8cdc03d415213c5 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:23:55.732851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3ee59-603b-4cd0-b073-2f1d4dd46ffa None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:55.733388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3ee59-603b-4cd0-b073-2f1d4dd46ffa None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:55.733388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3ee59-603b-4cd0-b073-2f1d4dd46ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:55.772736 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-34a3ee59-603b-4cd0-b073-2f1d4dd46ffa None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:55.774877 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-34a3ee59-603b-4cd0-b073-2f1d4dd46ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:55.931778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 843/1693] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:55.965529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085f423-2f5d-44f9-b043-25ad4a4eb10e admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:55.969624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8085f423-2f5d-44f9-b043-25ad4a4eb10e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:56.242084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085f423-2f5d-44f9-b043-25ad4a4eb10e admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:56.249214 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c6f84-9374-4fb4-b034-d808f8fd6b2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:56.249936 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c6f84-9374-4fb4-b034-d808f8fd6b2c service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:56.250715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c6f84-9374-4fb4-b034-d808f8fd6b2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:56.251015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c6f84-9374-4fb4-b034-d808f8fd6b2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:56.294788 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c6f84-9374-4fb4-b034-d808f8fd6b2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:56.296301 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c6f84-9374-4fb4-b034-d808f8fd6b2c service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:56.344037 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 851/1694] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:58.338723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15d5a0-15d1-4691-b896-82fe970f794f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:58.340625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15d5a0-15d1-4691-b896-82fe970f794f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:58.345703 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15d5a0-15d1-4691-b896-82fe970f794f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:58.445185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-0f15d5a0-15d1-4691-b896-82fe970f794f None None] MFA Rules not processed for user `6f852249c66a4f8986d7a63a957ec91c`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:58.446443 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0f15d5a0-15d1-4691-b896-82fe970f794f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:58.522660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:58.608481 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 844/1695] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:58.620757 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4dce7-9961-4934-a142-ecd3a9835df5 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:58.621343 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4dce7-9961-4934-a142-ecd3a9835df5 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:58.621435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4dce7-9961-4934-a142-ecd3a9835df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:58.662161 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-37c4dce7-9961-4934-a142-ecd3a9835df5 None None] MFA Rules not processed for user `6f852249c66a4f8986d7a63a957ec91c`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:58.665094 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-37c4dce7-9961-4934-a142-ecd3a9835df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:58.687651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 852/1696] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:58.705589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f15d5a0-15d1-4691-b896-82fe970f794f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:58.707752 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0f15d5a0-15d1-4691-b896-82fe970f794f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:58.828410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571445e-d98f-4e99-890d-d45451a251a0 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:58.828410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571445e-d98f-4e99-890d-d45451a251a0 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:58.828410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571445e-d98f-4e99-890d-d45451a251a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:58.887495 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-f571445e-d98f-4e99-890d-d45451a251a0 None None] MFA Rules not processed for user `c60ee64ca18e4930a14e3e17bc979ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:58.890953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f571445e-d98f-4e99-890d-d45451a251a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:58.896208 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f15d5a0-15d1-4691-b896-82fe970f794f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:58.909514 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9ddf2-94a1-48be-a886-a18e0d5f0f35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:58.910286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9ddf2-94a1-48be-a886-a18e0d5f0f35 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:58.910497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9ddf2-94a1-48be-a886-a18e0d5f0f35 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:58.910977 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9ddf2-94a1-48be-a886-a18e0d5f0f35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:58.926926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9ddf2-94a1-48be-a886-a18e0d5f0f35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:58.930951 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9ddf2-94a1-48be-a886-a18e0d5f0f35 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:58.960348 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 845/1697] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:58.961297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9ddf2-94a1-48be-a886-a18e0d5f0f35 service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:58.968423 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-84b9ddf2-94a1-48be-a886-a18e0d5f0f35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:58.991864 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9ddf2-94a1-48be-a886-a18e0d5f0f35 service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:59.004290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:59.005721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51947235-e95e-4921-9557-aa55100f3795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:59.006713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51947235-e95e-4921-9557-aa55100f3795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:59.007199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51947235-e95e-4921-9557-aa55100f3795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:59.007554 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51947235-e95e-4921-9557-aa55100f3795 admin admin] PATH_INFO: `/v3/projects/7c2c70a52022408fbc58c1a969f721b9` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:59.011885 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51947235-e95e-4921-9557-aa55100f3795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c2c70a52022408fbc58c1a969f721b9)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:59.013873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51947235-e95e-4921-9557-aa55100f3795 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:59.067428 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 853/1698] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:59.068435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571445e-d98f-4e99-890d-d45451a251a0 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:59.080643 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f571445e-d98f-4e99-890d-d45451a251a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:59.111559 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571445e-d98f-4e99-890d-d45451a251a0 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:59.133174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-51947235-e95e-4921-9557-aa55100f3795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c2c70a52022408fbc58c1a969f721b9', 'request_id': 'req-51947235-e95e-4921-9557-aa55100f3795'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:59.150889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-51947235-e95e-4921-9557-aa55100f3795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c2c70a52022408fbc58c1a969f721b9', 'request_id': 'req-51947235-e95e-4921-9557-aa55100f3795'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:59.163496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-51947235-e95e-4921-9557-aa55100f3795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c2c70a52022408fbc58c1a969f721b9', 'request_id': 'req-51947235-e95e-4921-9557-aa55100f3795'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:59.188226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-51947235-e95e-4921-9557-aa55100f3795 admin admin] The token cache is being invalidate because project 7c2c70a52022408fbc58c1a969f721b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:23:59.188349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-51947235-e95e-4921-9557-aa55100f3795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:59.189008 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-51947235-e95e-4921-9557-aa55100f3795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:59.194618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 846/1699] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:23:58 2022] DELETE /identity/v3/projects/7c2c70a52022408fbc58c1a969f721b9 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:59.194618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51947235-e95e-4921-9557-aa55100f3795 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:59.194618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-51947235-e95e-4921-9557-aa55100f3795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:59.278946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8d3b6-f7bc-4e3b-ba78-e2f70bfaf3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:59.291563 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8d3b6-f7bc-4e3b-ba78-e2f70bfaf3ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:59.291887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8d3b6-f7bc-4e3b-ba78-e2f70bfaf3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:59.292168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8d3b6-f7bc-4e3b-ba78-e2f70bfaf3ea admin admin] PATH_INFO: `/v3/users/b538ba9d25bb4d669c6b5255ad0528aa` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:59.294173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8d3b6-f7bc-4e3b-ba78-e2f70bfaf3ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=b538ba9d25bb4d669c6b5255ad0528aa)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:59.296264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8d3b6-f7bc-4e3b-ba78-e2f70bfaf3ea admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:59.324053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:59.390466 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51947235-e95e-4921-9557-aa55100f3795 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:59.400911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4ec20-3083-4305-a5cd-8cfe75de85ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:59.401137 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4ec20-3083-4305-a5cd-8cfe75de85ee service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:59.401576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4ec20-3083-4305-a5cd-8cfe75de85ee service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:59.402164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4ec20-3083-4305-a5cd-8cfe75de85ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:59.405300 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-16e4ec20-3083-4305-a5cd-8cfe75de85ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:59.454911 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-63b8d3b6-f7bc-4e3b-ba78-e2f70bfaf3ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b538ba9d25bb4d669c6b5255ad0528aa'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:59.475427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-63b8d3b6-f7bc-4e3b-ba78-e2f70bfaf3ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b538ba9d25bb4d669c6b5255ad0528aa'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:59.476630 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-63b8d3b6-f7bc-4e3b-ba78-e2f70bfaf3ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b538ba9d25bb4d669c6b5255ad0528aa'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:59.498856 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-63b8d3b6-f7bc-4e3b-ba78-e2f70bfaf3ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b538ba9d25bb4d669c6b5255ad0528aa'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:59.555360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-63b8d3b6-f7bc-4e3b-ba78-e2f70bfaf3ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b538ba9d25bb4d669c6b5255ad0528aa'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:59.578366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 854/1700] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:23:59 2022] DELETE /identity/v3/users/b538ba9d25bb4d669c6b5255ad0528aa => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:59.579053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc31f7c-3cf9-4067-a857-3e58f7f78a0f None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:59.582289 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc31f7c-3cf9-4067-a857-3e58f7f78a0f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:59.582835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc31f7c-3cf9-4067-a857-3e58f7f78a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:59.585307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4ec20-3083-4305-a5cd-8cfe75de85ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:59.587918 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4ec20-3083-4305-a5cd-8cfe75de85ee service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:59.601396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 847/1701] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:59.604242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4231e-fc0a-4b58-a814-5806f2dfb02a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:59.604900 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4231e-fc0a-4b58-a814-5806f2dfb02a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:59.605377 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4231e-fc0a-4b58-a814-5806f2dfb02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:59.641989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2ae4231e-fc0a-4b58-a814-5806f2dfb02a None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:59.643709 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2ae4231e-fc0a-4b58-a814-5806f2dfb02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:59.670673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-5dc31f7c-3cf9-4067-a857-3e58f7f78a0f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:59.673853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5dc31f7c-3cf9-4067-a857-3e58f7f78a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:59.811757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 848/1702] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:59.871623 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 855/1703] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:59.880277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cf1e6-a877-4a40-abb2-4513e79c772e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:59.880642 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cf1e6-a877-4a40-abb2-4513e79c772e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:59.880978 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cf1e6-a877-4a40-abb2-4513e79c772e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:59.935288 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc31f7c-3cf9-4067-a857-3e58f7f78a0f None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:59.940246 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e79cf1e6-a877-4a40-abb2-4513e79c772e None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:59.941236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5dc31f7c-3cf9-4067-a857-3e58f7f78a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:59.941477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e79cf1e6-a877-4a40-abb2-4513e79c772e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:59.950858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:59.985421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 849/1704] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:59.987259 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75387e25-e617-4d32-813b-acc1781b1685 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:59.987552 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75387e25-e617-4d32-813b-acc1781b1685 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:59.987838 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75387e25-e617-4d32-813b-acc1781b1685 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:59.994418 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 850/1705] 10.222.0.48 () {54 vars in 912 bytes} [Mon Aug 1 15:23:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:23:59.995385 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75387e25-e617-4d32-813b-acc1781b1685 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:00.019102 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75387e25-e617-4d32-813b-acc1781b1685 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:00.038586 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314991e-d436-458d-a8d1-151ecdb26cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:00.041355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314991e-d436-458d-a8d1-151ecdb26cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:00.041355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314991e-d436-458d-a8d1-151ecdb26cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:00.041355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314991e-d436-458d-a8d1-151ecdb26cd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:00.042523 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314991e-d436-458d-a8d1-151ecdb26cd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:24:00.044061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314991e-d436-458d-a8d1-151ecdb26cd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:00.046132 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314991e-d436-458d-a8d1-151ecdb26cd8 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:00.067160 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 851/1706] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:24:00.069229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f8b63-4054-4266-b600-adddfdd284b3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:00.069526 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f8b63-4054-4266-b600-adddfdd284b3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:00.069890 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f8b63-4054-4266-b600-adddfdd284b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:00.138670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc31f7c-3cf9-4067-a857-3e58f7f78a0f None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:00.143636 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f9373-6edb-4dc1-8ea7-04199384aa40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:00.144766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f9373-6edb-4dc1-8ea7-04199384aa40 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:00.145376 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f9373-6edb-4dc1-8ea7-04199384aa40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:00.145825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f9373-6edb-4dc1-8ea7-04199384aa40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:00.157809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a36f8b63-4054-4266-b600-adddfdd284b3 None None] MFA Rules not processed for user `5ef6577b10f7426a925d594e627f27cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:00.160895 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f9373-6edb-4dc1-8ea7-04199384aa40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:00.162647 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a36f8b63-4054-4266-b600-adddfdd284b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:00.163316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f9373-6edb-4dc1-8ea7-04199384aa40 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:00.193498 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 856/1707] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:23:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:00.194711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f9373-6edb-4dc1-8ea7-04199384aa40 service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:00.204922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 852/1708] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 15:24:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:00.219589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f9373-6edb-4dc1-8ea7-04199384aa40 service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:00.231030 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f8b63-4054-4266-b600-adddfdd284b3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:00.235233 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca48eff-42bf-411b-9b4e-dcd368ec3e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:00.235606 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca48eff-42bf-411b-9b4e-dcd368ec3e2a admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:00.235904 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca48eff-42bf-411b-9b4e-dcd368ec3e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:00.236239 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca48eff-42bf-411b-9b4e-dcd368ec3e2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:00.236782 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a36f8b63-4054-4266-b600-adddfdd284b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:00.237069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca48eff-42bf-411b-9b4e-dcd368ec3e2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:00.243386 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca48eff-42bf-411b-9b4e-dcd368ec3e2a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:00.278299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f8b63-4054-4266-b600-adddfdd284b3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:00.335668 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 857/1709] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:00 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:00.343696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca48eff-42bf-411b-9b4e-dcd368ec3e2a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:00.364716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d4ac5-5d33-49cd-90d7-97ddc4f10e0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:00.365761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d4ac5-5d33-49cd-90d7-97ddc4f10e0d service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:00.366264 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d4ac5-5d33-49cd-90d7-97ddc4f10e0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:00.368808 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d4ac5-5d33-49cd-90d7-97ddc4f10e0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:00.395914 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d4ac5-5d33-49cd-90d7-97ddc4f10e0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:00.398728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d4ac5-5d33-49cd-90d7-97ddc4f10e0d service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:00.415413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 853/1710] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:00.562531 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca48eff-42bf-411b-9b4e-dcd368ec3e2a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:00.576021 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676bb2b-d638-4fdf-a54f-cce142adf32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:00.576685 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676bb2b-d638-4fdf-a54f-cce142adf32e admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:00.577930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676bb2b-d638-4fdf-a54f-cce142adf32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:00.578234 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c676bb2b-d638-4fdf-a54f-cce142adf32e admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:00.579091 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676bb2b-d638-4fdf-a54f-cce142adf32e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:00.581094 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c676bb2b-d638-4fdf-a54f-cce142adf32e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:00.615113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 858/1711] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:00 2022] POST /identity/v3/users => generated 287 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:24:00.621181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d4ac5-5d33-49cd-90d7-97ddc4f10e0d service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:00.656464 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d4ac5-5d33-49cd-90d7-97ddc4f10e0d service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:00.675661 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a2abe-2e1e-4cd0-a8f9-e58447179b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:00.677256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a2abe-2e1e-4cd0-a8f9-e58447179b22 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:00.678825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a2abe-2e1e-4cd0-a8f9-e58447179b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:00.679502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a2abe-2e1e-4cd0-a8f9-e58447179b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:00.680901 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a2abe-2e1e-4cd0-a8f9-e58447179b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:00.684131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a2abe-2e1e-4cd0-a8f9-e58447179b22 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:00.761055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676bb2b-d638-4fdf-a54f-cce142adf32e admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:00.761676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 854/1712] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:00.785005 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c676bb2b-d638-4fdf-a54f-cce142adf32e admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:00.798108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d327b3-7f2b-420e-81ee-a6f6dcd73a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:00.799083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d327b3-7f2b-420e-81ee-a6f6dcd73a76 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:00.799532 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d327b3-7f2b-420e-81ee-a6f6dcd73a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:00.800555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d327b3-7f2b-420e-81ee-a6f6dcd73a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:00.801190 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d327b3-7f2b-420e-81ee-a6f6dcd73a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:00.804040 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d327b3-7f2b-420e-81ee-a6f6dcd73a76 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:00.825835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 859/1713] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:00.828192 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a2abe-2e1e-4cd0-a8f9-e58447179b22 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:00.844170 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a2abe-2e1e-4cd0-a8f9-e58447179b22 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:00.851031 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6f8ca-c49f-4480-ae2c-81657643e874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:00.851880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6f8ca-c49f-4480-ae2c-81657643e874 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:00.852266 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6f8ca-c49f-4480-ae2c-81657643e874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:00.852732 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6f8ca-c49f-4480-ae2c-81657643e874 admin admin] PATH_INFO: `/v3/projects/087f9994cf454b1aad27728873b37a8f/users/5f47a5a90a5c4e56ac373388657e5e1f/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:00.858671 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:00.915569 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b6f8ca-c49f-4480-ae2c-81657643e874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087f9994cf454b1aad27728873b37a8f, user_id=5f47a5a90a5c4e56ac373388657e5e1f, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:00.925094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b6f8ca-c49f-4480-ae2c-81657643e874 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:00.990478 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 855/1714] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:00 2022] PUT /identity/v3/projects/087f9994cf454b1aad27728873b37a8f/users/5f47a5a90a5c4e56ac373388657e5e1f/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:00.995180 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e6f57-1718-48ae-9a35-4f83d369ffae None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:00.995484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e6f57-1718-48ae-9a35-4f83d369ffae None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:00.995835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e6f57-1718-48ae-9a35-4f83d369ffae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:01.071823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-6c7e6f57-1718-48ae-9a35-4f83d369ffae None None] MFA Rules not processed for user `5f47a5a90a5c4e56ac373388657e5e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:01.073688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6c7e6f57-1718-48ae-9a35-4f83d369ffae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:01.201295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:01.245068 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6f8ca-c49f-4480-ae2c-81657643e874 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:01.246548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 860/1715] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:01.468505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847f373-6289-4508-a054-300e56b1f04f None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:01.468786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847f373-6289-4508-a054-300e56b1f04f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:01.469112 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847f373-6289-4508-a054-300e56b1f04f None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:01.470865 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 861/1716] 10.222.0.48 () {54 vars in 961 bytes} [Mon Aug 1 15:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:24:01.477753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bd43a-1a1e-416c-b889-208b4f3fbbed None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:01.478144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bd43a-1a1e-416c-b889-208b4f3fbbed None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:01.478415 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bd43a-1a1e-416c-b889-208b4f3fbbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:01.600481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-345bd43a-1a1e-416c-b889-208b4f3fbbed None None] MFA Rules not processed for user `098d8d7e35804ccc931ddfba07f688a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:01.601753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-345bd43a-1a1e-416c-b889-208b4f3fbbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:01.654997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6f8ca-c49f-4480-ae2c-81657643e874 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:01.663028 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21339fe1-4a2a-4350-ba22-17adc349673d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:01.664417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21339fe1-4a2a-4350-ba22-17adc349673d admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:01.665098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21339fe1-4a2a-4350-ba22-17adc349673d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:01.665513 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21339fe1-4a2a-4350-ba22-17adc349673d admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:01.666869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21339fe1-4a2a-4350-ba22-17adc349673d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:01.670879 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21339fe1-4a2a-4350-ba22-17adc349673d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:01.740793 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 856/1717] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:01 2022] POST /identity/v3/projects => generated 369 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:01.746833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21339fe1-4a2a-4350-ba22-17adc349673d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:01.939317 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 862/1718] 10.222.0.48 () {58 vars in 1058 bytes} [Mon Aug 1 15:24:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:01.941717 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7581e-b5cc-4b41-abf5-94a5985410ec None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:01.942105 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7581e-b5cc-4b41-abf5-94a5985410ec None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:01.942364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7581e-b5cc-4b41-abf5-94a5985410ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:01.964064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21339fe1-4a2a-4350-ba22-17adc349673d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:01.972865 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-c5b7581e-b5cc-4b41-abf5-94a5985410ec None None] MFA Rules not processed for user `6f852249c66a4f8986d7a63a957ec91c`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:01.974392 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c5b7581e-b5cc-4b41-abf5-94a5985410ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:01.976252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bce54b-34d7-4100-b25a-af3d0f476ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:01.977080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bce54b-34d7-4100-b25a-af3d0f476ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:01.977504 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bce54b-34d7-4100-b25a-af3d0f476ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:01.977970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bce54b-34d7-4100-b25a-af3d0f476ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:01.978807 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bce54b-34d7-4100-b25a-af3d0f476ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:01.980969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bce54b-34d7-4100-b25a-af3d0f476ad5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:02.034607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 857/1719] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:01 2022] POST /identity/v3/users => generated 291 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:02.034607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bce54b-34d7-4100-b25a-af3d0f476ad5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:02.175631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 863/1720] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:02.175631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804ece4-4ca9-43a5-98c8-a1f950d52d24 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:02.175631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804ece4-4ca9-43a5-98c8-a1f950d52d24 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:02.175631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804ece4-4ca9-43a5-98c8-a1f950d52d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:02.243590 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:02.244886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-3804ece4-4ca9-43a5-98c8-a1f950d52d24 None None] MFA Rules not processed for user `d79f95d9699b4b0393e14a9212986ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:02.263407 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bce54b-34d7-4100-b25a-af3d0f476ad5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:02.266407 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-3804ece4-4ca9-43a5-98c8-a1f950d52d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:02.286846 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12fbc7-d8e8-4192-a1a3-86b58a6ce74c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '098d8d7e35804ccc931ddfba07f688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:02.287906 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12fbc7-d8e8-4192-a1a3-86b58a6ce74c service placement] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:02.290727 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12fbc7-d8e8-4192-a1a3-86b58a6ce74c service placement] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:02.291228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12fbc7-d8e8-4192-a1a3-86b58a6ce74c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:02.298717 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1c12fbc7-d8e8-4192-a1a3-86b58a6ce74c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:02.371566 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12fbc7-d8e8-4192-a1a3-86b58a6ce74c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:02.379662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12fbc7-d8e8-4192-a1a3-86b58a6ce74c service placement] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:02.421879 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 858/1721] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:02.422933 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12fbc7-d8e8-4192-a1a3-86b58a6ce74c service placement] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:02.457150 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12fbc7-d8e8-4192-a1a3-86b58a6ce74c service placement] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:02.477157 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87f0aa-d0d1-4740-b9ac-9ec7b719ff9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:02.478038 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87f0aa-d0d1-4740-b9ac-9ec7b719ff9c admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:02.478329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87f0aa-d0d1-4740-b9ac-9ec7b719ff9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:02.478677 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87f0aa-d0d1-4740-b9ac-9ec7b719ff9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:02.479417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87f0aa-d0d1-4740-b9ac-9ec7b719ff9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:02.481211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87f0aa-d0d1-4740-b9ac-9ec7b719ff9c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:02.521050 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 859/1722] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:02.524196 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ba6f4-eab4-4b71-9b35-61282b36e685 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:02.524402 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ba6f4-eab4-4b71-9b35-61282b36e685 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:02.525036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ba6f4-eab4-4b71-9b35-61282b36e685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:02.558678 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 864/1723] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:02.572300 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804ece4-4ca9-43a5-98c8-a1f950d52d24 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:02.574297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-3804ece4-4ca9-43a5-98c8-a1f950d52d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:02.618907 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-635ba6f4-eab4-4b71-9b35-61282b36e685 None None] MFA Rules not processed for user `e6945b8e606f40f1b79bd4aab96a7f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:02.628176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-635ba6f4-eab4-4b71-9b35-61282b36e685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:02.628405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804ece4-4ca9-43a5-98c8-a1f950d52d24 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:02.640406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:02.642061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:02.642353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:02.642632 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29 admin admin] PATH_INFO: `/v3/projects/85e24ab0539d4acc8447b8b898124707` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:02.643967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85e24ab0539d4acc8447b8b898124707)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:02.664169 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:02.749090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:02.891986 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85e24ab0539d4acc8447b8b898124707', 'request_id': 'req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:02.945105 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85e24ab0539d4acc8447b8b898124707', 'request_id': 'req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:02.985245 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85e24ab0539d4acc8447b8b898124707', 'request_id': 'req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:03.023615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 860/1724] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:03.025121 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ba6f4-eab4-4b71-9b35-61282b36e685 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:03.037309 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29 admin admin] The token cache is being invalidate because project 85e24ab0539d4acc8447b8b898124707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:03.037810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:03.038803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:03.042974 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 865/1725] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:02 2022] DELETE /identity/v3/projects/85e24ab0539d4acc8447b8b898124707 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:03.047308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:03.055699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ba6f4-eab4-4b71-9b35-61282b36e685 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:03.067558 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:03.174160 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e0d08-8078-473c-881d-bf69a584f274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:03.175011 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e0d08-8078-473c-881d-bf69a584f274 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:03.175399 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e0d08-8078-473c-881d-bf69a584f274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:03.175777 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e0d08-8078-473c-881d-bf69a584f274 admin admin] PATH_INFO: `/v3/projects/d2aff9beb78a4e7cacdcf0e2958ce57c/users/09c50cffdf694da39d436964b9052439/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:03.212326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e0d08-8078-473c-881d-bf69a584f274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2aff9beb78a4e7cacdcf0e2958ce57c, user_id=09c50cffdf694da39d436964b9052439, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:03.215634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e0d08-8078-473c-881d-bf69a584f274 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:03.255609 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 861/1726] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:03 2022] PUT /identity/v3/projects/d2aff9beb78a4e7cacdcf0e2958ce57c/users/09c50cffdf694da39d436964b9052439/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:03.258653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b939b-cb1e-4078-9ef6-156e19f19c69 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:03.259329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b939b-cb1e-4078-9ef6-156e19f19c69 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:03.260023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b939b-cb1e-4078-9ef6-156e19f19c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:03.293324 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-628b939b-cb1e-4078-9ef6-156e19f19c69 None None] MFA Rules not processed for user `e6945b8e606f40f1b79bd4aab96a7f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:03.296599 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-628b939b-cb1e-4078-9ef6-156e19f19c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:03.346438 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d3d4e-8cb3-4805-b565-af62eccb3b29 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:03.445048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:03.473524 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 862/1727] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:03.480122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b939b-cb1e-4078-9ef6-156e19f19c69 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:03.483217 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-628b939b-cb1e-4078-9ef6-156e19f19c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:03.593728 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff816e-3ebb-4a6d-82b7-7a6e6bbe7fbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:03.594444 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff816e-3ebb-4a6d-82b7-7a6e6bbe7fbb service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:03.594722 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff816e-3ebb-4a6d-82b7-7a6e6bbe7fbb service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:03.595148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff816e-3ebb-4a6d-82b7-7a6e6bbe7fbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:03.597892 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-13ff816e-3ebb-4a6d-82b7-7a6e6bbe7fbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:03.661247 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b939b-cb1e-4078-9ef6-156e19f19c69 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:03.676313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d02ca-61d6-4a8f-b829-d83bec31785b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:03.679082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d02ca-61d6-4a8f-b829-d83bec31785b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:03.679607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d02ca-61d6-4a8f-b829-d83bec31785b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:03.680023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d02ca-61d6-4a8f-b829-d83bec31785b admin admin] PATH_INFO: `/v3/users/fc8a9e1c993a4232b220ca70898b826c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:03.683675 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d02ca-61d6-4a8f-b829-d83bec31785b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc8a9e1c993a4232b220ca70898b826c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:03.688414 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d02ca-61d6-4a8f-b829-d83bec31785b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:03.716887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:03.771512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff816e-3ebb-4a6d-82b7-7a6e6bbe7fbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:03.774067 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff816e-3ebb-4a6d-82b7-7a6e6bbe7fbb service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:03.799938 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 866/1728] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 757 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:03.800761 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff816e-3ebb-4a6d-82b7-7a6e6bbe7fbb service nova] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:03.804654 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-13ff816e-3ebb-4a6d-82b7-7a6e6bbe7fbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:03.837685 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff816e-3ebb-4a6d-82b7-7a6e6bbe7fbb service nova] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:03.847746 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae389abb-7851-42cc-8a96-a95ca47094da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:03.848408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae389abb-7851-42cc-8a96-a95ca47094da admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:03.848689 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae389abb-7851-42cc-8a96-a95ca47094da admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:03.848962 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae389abb-7851-42cc-8a96-a95ca47094da admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:03.854028 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d40d02ca-61d6-4a8f-b829-d83bec31785b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc8a9e1c993a4232b220ca70898b826c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:03.854544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae389abb-7851-42cc-8a96-a95ca47094da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:03.856487 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae389abb-7851-42cc-8a96-a95ca47094da admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:03.857533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d40d02ca-61d6-4a8f-b829-d83bec31785b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc8a9e1c993a4232b220ca70898b826c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:03.895009 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 867/1729] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:03.899829 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae389abb-7851-42cc-8a96-a95ca47094da admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:03.918131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d40d02ca-61d6-4a8f-b829-d83bec31785b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc8a9e1c993a4232b220ca70898b826c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:03.923891 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae389abb-7851-42cc-8a96-a95ca47094da admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:03.936588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67fb0f-d601-445d-9d89-924e350360d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:03.937369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67fb0f-d601-445d-9d89-924e350360d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:03.937651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67fb0f-d601-445d-9d89-924e350360d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:03.937958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67fb0f-d601-445d-9d89-924e350360d6 admin admin] PATH_INFO: `/v3/projects/d2aff9beb78a4e7cacdcf0e2958ce57c/users/09c50cffdf694da39d436964b9052439/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:03.945327 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c67fb0f-d601-445d-9d89-924e350360d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2aff9beb78a4e7cacdcf0e2958ce57c, user_id=09c50cffdf694da39d436964b9052439, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:03.951147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d40d02ca-61d6-4a8f-b829-d83bec31785b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc8a9e1c993a4232b220ca70898b826c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:03.953124 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c67fb0f-d601-445d-9d89-924e350360d6 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:03.968876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d40d02ca-61d6-4a8f-b829-d83bec31785b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc8a9e1c993a4232b220ca70898b826c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:03.981329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 863/1730] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:03 2022] DELETE /identity/v3/users/fc8a9e1c993a4232b220ca70898b826c => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:03.984759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 868/1731] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:03 2022] PUT /identity/v3/projects/d2aff9beb78a4e7cacdcf0e2958ce57c/users/09c50cffdf694da39d436964b9052439/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:03.997522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48771a3a-7689-4123-aaf2-da934a9f7370 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:03.997859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48771a3a-7689-4123-aaf2-da934a9f7370 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:03.999206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67fb0f-d601-445d-9d89-924e350360d6 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:04.000526 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48771a3a-7689-4123-aaf2-da934a9f7370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:04.002436 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5c67fb0f-d601-445d-9d89-924e350360d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:04.142990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-48771a3a-7689-4123-aaf2-da934a9f7370 None None] MFA Rules not processed for user `09c50cffdf694da39d436964b9052439`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:04.144198 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-48771a3a-7689-4123-aaf2-da934a9f7370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:04.325878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67fb0f-d601-445d-9d89-924e350360d6 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:04.408345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:04.436685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 864/1732] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:03 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:04.446154 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48771a3a-7689-4123-aaf2-da934a9f7370 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:04.455265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-48771a3a-7689-4123-aaf2-da934a9f7370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:04.739733 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d60af-de23-456b-b5de-69682e455aaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:04.743293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d60af-de23-456b-b5de-69682e455aaa service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:04.748276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d60af-de23-456b-b5de-69682e455aaa service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:04.748496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d60af-de23-456b-b5de-69682e455aaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:04.760997 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6d0d60af-de23-456b-b5de-69682e455aaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:04.779894 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48771a3a-7689-4123-aaf2-da934a9f7370 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:04.790062 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc9f0e-47bd-4c3d-b0c8-3c0a04651e52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:04.790922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc9f0e-47bd-4c3d-b0c8-3c0a04651e52 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:04.791346 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc9f0e-47bd-4c3d-b0c8-3c0a04651e52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:04.791733 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc9f0e-47bd-4c3d-b0c8-3c0a04651e52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:04.828138 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc9f0e-47bd-4c3d-b0c8-3c0a04651e52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:04.832494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc9f0e-47bd-4c3d-b0c8-3c0a04651e52 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:04.893462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 865/1733] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:04 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:04.999713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:05.053138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d60af-de23-456b-b5de-69682e455aaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:05.055212 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d60af-de23-456b-b5de-69682e455aaa service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:05.114986 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 869/1734] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 1116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:05.173826 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc9f0e-47bd-4c3d-b0c8-3c0a04651e52 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:05.186026 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc9f0e-47bd-4c3d-b0c8-3c0a04651e52 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:05.205943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7baf0-0535-44aa-b0eb-1401ba75c4a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:05.206772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7baf0-0535-44aa-b0eb-1401ba75c4a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:05.207159 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7baf0-0535-44aa-b0eb-1401ba75c4a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:05.207554 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7baf0-0535-44aa-b0eb-1401ba75c4a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:05.215663 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-06e7baf0-0535-44aa-b0eb-1401ba75c4a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:05.495106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7baf0-0535-44aa-b0eb-1401ba75c4a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:05.496499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7baf0-0535-44aa-b0eb-1401ba75c4a7 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:05.521169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 866/1735] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:06.279452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318029b-4813-47e7-87f0-f52b43f3d2e9 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:06.280270 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318029b-4813-47e7-87f0-f52b43f3d2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:06.280697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318029b-4813-47e7-87f0-f52b43f3d2e9 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:06.282580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 870/1736] 10.222.0.48 () {54 vars in 912 bytes} [Mon Aug 1 15:24:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:24:06.289334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d87beb-4bae-44a2-843a-904467093ea6 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:06.289938 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d87beb-4bae-44a2-843a-904467093ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:06.290028 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d87beb-4bae-44a2-843a-904467093ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:06.335941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054bc3d-e14c-4e4e-8b25-bb05e58d541b None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:06.336213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054bc3d-e14c-4e4e-8b25-bb05e58d541b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:06.336489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054bc3d-e14c-4e4e-8b25-bb05e58d541b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:06.339403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-00d87beb-4bae-44a2-843a-904467093ea6 None None] MFA Rules not processed for user `5ef6577b10f7426a925d594e627f27cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:06.342615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-00d87beb-4bae-44a2-843a-904467093ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:06.483244 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 867/1737] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 15:24:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:06.496833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-8054bc3d-e14c-4e4e-8b25-bb05e58d541b None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:06.499163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8054bc3d-e14c-4e4e-8b25-bb05e58d541b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:06.514421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d87beb-4bae-44a2-843a-904467093ea6 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:06.515699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-00d87beb-4bae-44a2-843a-904467093ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:06.529871 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d87beb-4bae-44a2-843a-904467093ea6 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:06.534885 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6aad9-afdf-465a-9f7e-52e8068d1609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:06.535609 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6aad9-afdf-465a-9f7e-52e8068d1609 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:06.536167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6aad9-afdf-465a-9f7e-52e8068d1609 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:06.536438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6aad9-afdf-465a-9f7e-52e8068d1609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:06.546849 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6aad9-afdf-465a-9f7e-52e8068d1609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:06.555953 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6aad9-afdf-465a-9f7e-52e8068d1609 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:06.566264 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 868/1738] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:24:06.568306 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a090e48-38c1-45ae-a34f-1908daa20a52 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:06.568693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a090e48-38c1-45ae-a34f-1908daa20a52 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:06.569145 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a090e48-38c1-45ae-a34f-1908daa20a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:06.609576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3a090e48-38c1-45ae-a34f-1908daa20a52 None None] MFA Rules not processed for user `93d86ae4501e43d59d172d2e7acfdd54`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:06.610862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3a090e48-38c1-45ae-a34f-1908daa20a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:06.713610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:06.713858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 871/1739] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 15:24:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:06.840155 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 869/1740] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:06.856050 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054bc3d-e14c-4e4e-8b25-bb05e58d541b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:06.857500 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8054bc3d-e14c-4e4e-8b25-bb05e58d541b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:06.896727 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054bc3d-e14c-4e4e-8b25-bb05e58d541b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:06.922233 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f729507-fcc8-4709-8cab-335d39e4e2f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:06.927983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f729507-fcc8-4709-8cab-335d39e4e2f8 service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:06.928478 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f729507-fcc8-4709-8cab-335d39e4e2f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:06.928766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f729507-fcc8-4709-8cab-335d39e4e2f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:06.970423 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f729507-fcc8-4709-8cab-335d39e4e2f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:06.982910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f729507-fcc8-4709-8cab-335d39e4e2f8 service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:07.031310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 872/1741] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:24:07.086334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a090e48-38c1-45ae-a34f-1908daa20a52 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:07.087869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3a090e48-38c1-45ae-a34f-1908daa20a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:07.446237 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a090e48-38c1-45ae-a34f-1908daa20a52 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:07.465499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf88644-3503-462b-997c-c0f61e17affd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:07.466194 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf88644-3503-462b-997c-c0f61e17affd service swift] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:07.466466 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf88644-3503-462b-997c-c0f61e17affd service swift] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:07.466893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf88644-3503-462b-997c-c0f61e17affd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:07.518022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf88644-3503-462b-997c-c0f61e17affd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:07.519696 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf88644-3503-462b-997c-c0f61e17affd service swift] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:07.547393 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 870/1742] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:24:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:24:07.812323 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f729507-fcc8-4709-8cab-335d39e4e2f8 service nova] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:07.813868 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1f729507-fcc8-4709-8cab-335d39e4e2f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:08.259063 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f729507-fcc8-4709-8cab-335d39e4e2f8 service nova] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:08.296595 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8dbf20-c107-4ff4-8501-59f320c113a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:08.297369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dbf20-c107-4ff4-8501-59f320c113a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:08.297698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dbf20-c107-4ff4-8501-59f320c113a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:08.300546 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dbf20-c107-4ff4-8501-59f320c113a1 admin admin] PATH_INFO: `/v3/projects/0cb3cb6d940d4dcc9b4a2e8417ecd9a6` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:08.301504 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8dbf20-c107-4ff4-8501-59f320c113a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cb3cb6d940d4dcc9b4a2e8417ecd9a6)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:08.303988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8dbf20-c107-4ff4-8501-59f320c113a1 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:08.562190 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4f8dbf20-c107-4ff4-8501-59f320c113a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cb3cb6d940d4dcc9b4a2e8417ecd9a6', 'request_id': 'req-4f8dbf20-c107-4ff4-8501-59f320c113a1'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:08.569085 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:08.582723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4f8dbf20-c107-4ff4-8501-59f320c113a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cb3cb6d940d4dcc9b4a2e8417ecd9a6', 'request_id': 'req-4f8dbf20-c107-4ff4-8501-59f320c113a1'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:08.607661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4f8dbf20-c107-4ff4-8501-59f320c113a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cb3cb6d940d4dcc9b4a2e8417ecd9a6', 'request_id': 'req-4f8dbf20-c107-4ff4-8501-59f320c113a1'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:08.651050 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4f8dbf20-c107-4ff4-8501-59f320c113a1 admin admin] The token cache is being invalidate because project 0cb3cb6d940d4dcc9b4a2e8417ecd9a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:08.651638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4f8dbf20-c107-4ff4-8501-59f320c113a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:08.654136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4f8dbf20-c107-4ff4-8501-59f320c113a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:08.676579 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 873/1743] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:07 2022] DELETE /identity/v3/projects/0cb3cb6d940d4dcc9b4a2e8417ecd9a6 => generated 0 bytes in 865 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:08.681807 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c331250-fcc8-4bd3-938d-abf5eb797bd8 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:08.682606 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c331250-fcc8-4bd3-938d-abf5eb797bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:08.682606 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c331250-fcc8-4bd3-938d-abf5eb797bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:08.751771 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-0c331250-fcc8-4bd3-938d-abf5eb797bd8 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:08.753410 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0c331250-fcc8-4bd3-938d-abf5eb797bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:08.794576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef8568-814f-4005-9988-c2f9c9a94894 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:08.794862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef8568-814f-4005-9988-c2f9c9a94894 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:08.795136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef8568-814f-4005-9988-c2f9c9a94894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:08.883667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-fdef8568-814f-4005-9988-c2f9c9a94894 None None] MFA Rules not processed for user `c60ee64ca18e4930a14e3e17bc979ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:08.887716 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-fdef8568-814f-4005-9988-c2f9c9a94894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:09.093458 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 874/1744] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:24:09.142853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 871/1745] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:09.152188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ee72b-3d2a-4e8b-9409-862053f10d06 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:09.153147 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ee72b-3d2a-4e8b-9409-862053f10d06 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:09.153736 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ee72b-3d2a-4e8b-9409-862053f10d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:09.276327 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-dc4ee72b-3d2a-4e8b-9409-862053f10d06 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:09.304889 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-dc4ee72b-3d2a-4e8b-9409-862053f10d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:09.380013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 875/1746] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:09.380204 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c331250-fcc8-4bd3-938d-abf5eb797bd8 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:09.433120 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c331250-fcc8-4bd3-938d-abf5eb797bd8 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:09.475587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a6b27-0109-40d5-913d-e8a571d2dd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:09.476542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a6b27-0109-40d5-913d-e8a571d2dd0e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:09.476969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a6b27-0109-40d5-913d-e8a571d2dd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:09.477385 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a6b27-0109-40d5-913d-e8a571d2dd0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:09.478338 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a6b27-0109-40d5-913d-e8a571d2dd0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:24:09.478931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a6b27-0109-40d5-913d-e8a571d2dd0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:09.483366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a6b27-0109-40d5-913d-e8a571d2dd0e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:09.530754 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 872/1747] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:24:09.533796 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ee72b-3d2a-4e8b-9409-862053f10d06 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:09.548987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ee72b-3d2a-4e8b-9409-862053f10d06 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:09.612758 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb75509-eaf5-4eba-b378-f3ca5ecb8c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:09.613476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb75509-eaf5-4eba-b378-f3ca5ecb8c8f admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:09.613838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb75509-eaf5-4eba-b378-f3ca5ecb8c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:09.622098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb75509-eaf5-4eba-b378-f3ca5ecb8c8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:09.631325 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb75509-eaf5-4eba-b378-f3ca5ecb8c8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:09.648103 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb75509-eaf5-4eba-b378-f3ca5ecb8c8f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:09.797939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a6b27-0109-40d5-913d-e8a571d2dd0e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:09.798535 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 876/1748] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:09 2022] POST /identity/v3/projects => generated 381 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:09.821807 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a6b27-0109-40d5-913d-e8a571d2dd0e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:09.951360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:10.153444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1d6c7-a36e-4ac1-9c48-73f58148106f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:10.155010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1d6c7-a36e-4ac1-9c48-73f58148106f admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:10.158380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1d6c7-a36e-4ac1-9c48-73f58148106f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:10.160325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1d6c7-a36e-4ac1-9c48-73f58148106f admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:10.164958 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1d6c7-a36e-4ac1-9c48-73f58148106f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:10.179475 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1d6c7-a36e-4ac1-9c48-73f58148106f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:10.276990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 873/1749] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:09 2022] POST /identity/v3/users => generated 291 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:10.281294 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb75509-eaf5-4eba-b378-f3ca5ecb8c8f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:10.316516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb75509-eaf5-4eba-b378-f3ca5ecb8c8f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:10.330904 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72e1c8-d199-45f8-b243-37a5005dfc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:10.332313 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72e1c8-d199-45f8-b243-37a5005dfc42 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:10.334461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72e1c8-d199-45f8-b243-37a5005dfc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:10.335393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72e1c8-d199-45f8-b243-37a5005dfc42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:10.337695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72e1c8-d199-45f8-b243-37a5005dfc42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:10.342559 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72e1c8-d199-45f8-b243-37a5005dfc42 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:10.391884 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 877/1750] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:10.393153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1d6c7-a36e-4ac1-9c48-73f58148106f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:10.430224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1d6c7-a36e-4ac1-9c48-73f58148106f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:10.450133 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8c54c-535b-48f1-a1cd-d4c46260899e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:10.450805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8c54c-535b-48f1-a1cd-d4c46260899e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:10.451083 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8c54c-535b-48f1-a1cd-d4c46260899e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:10.451362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8c54c-535b-48f1-a1cd-d4c46260899e admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:10.452098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8c54c-535b-48f1-a1cd-d4c46260899e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:10.486216 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8c54c-535b-48f1-a1cd-d4c46260899e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:10.553269 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 874/1751] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:10.567330 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72e1c8-d199-45f8-b243-37a5005dfc42 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:10.724131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72e1c8-d199-45f8-b243-37a5005dfc42 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:10.759042 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d4704-74f3-4338-8194-082734fff474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:10.759879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d4704-74f3-4338-8194-082734fff474 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:10.760159 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d4704-74f3-4338-8194-082734fff474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:10.760435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d4704-74f3-4338-8194-082734fff474 admin admin] PATH_INFO: `/v3/projects/4c922afd0c304da0b219ad0265c3ee6a/users/fb4d21e643ef4f78bc471461c76936f1/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:10.883353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d4704-74f3-4338-8194-082734fff474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c922afd0c304da0b219ad0265c3ee6a, user_id=fb4d21e643ef4f78bc471461c76936f1, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:10.886301 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d4704-74f3-4338-8194-082734fff474 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:10.971530 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 878/1752] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:10 2022] PUT /identity/v3/projects/4c922afd0c304da0b219ad0265c3ee6a/users/fb4d21e643ef4f78bc471461c76936f1/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:10.983507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de0678-5a0d-4414-b39b-b7225817108a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:10.984070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de0678-5a0d-4414-b39b-b7225817108a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:10.984070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de0678-5a0d-4414-b39b-b7225817108a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:11.080600 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a9de0678-5a0d-4414-b39b-b7225817108a None None] MFA Rules not processed for user `fb4d21e643ef4f78bc471461c76936f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:11.083797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a9de0678-5a0d-4414-b39b-b7225817108a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:11.200948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 875/1753] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:11.205468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d4704-74f3-4338-8194-082734fff474 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:11.294402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d4704-74f3-4338-8194-082734fff474 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:11.312526 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101adf88-7342-40a2-a970-612b45451cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:11.313511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101adf88-7342-40a2-a970-612b45451cad admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:11.314051 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101adf88-7342-40a2-a970-612b45451cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:11.314541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101adf88-7342-40a2-a970-612b45451cad admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:11.315500 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101adf88-7342-40a2-a970-612b45451cad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:11.319672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101adf88-7342-40a2-a970-612b45451cad admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:11.361490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 879/1754] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:11 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:11.365629 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de0678-5a0d-4414-b39b-b7225817108a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:11.378630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de0678-5a0d-4414-b39b-b7225817108a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:11.462322 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea371e4-4a41-4427-bdfc-ec0d1b600b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:11.469431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea371e4-4a41-4427-bdfc-ec0d1b600b64 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:11.469922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea371e4-4a41-4427-bdfc-ec0d1b600b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:11.470321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea371e4-4a41-4427-bdfc-ec0d1b600b64 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:11.472986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea371e4-4a41-4427-bdfc-ec0d1b600b64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:11.489477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea371e4-4a41-4427-bdfc-ec0d1b600b64 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:11.535798 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 876/1755] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:11 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:11.546203 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101adf88-7342-40a2-a970-612b45451cad admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:11.558604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101adf88-7342-40a2-a970-612b45451cad admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:11.565715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5bab8-8efe-4c86-8d1f-6081b0c39327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:11.566288 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5bab8-8efe-4c86-8d1f-6081b0c39327 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:11.566677 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5bab8-8efe-4c86-8d1f-6081b0c39327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:11.567360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5bab8-8efe-4c86-8d1f-6081b0c39327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:11.567963 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5bab8-8efe-4c86-8d1f-6081b0c39327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:11.569994 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5bab8-8efe-4c86-8d1f-6081b0c39327 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:11.573433 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:11.592202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 880/1756] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:11.593524 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea371e4-4a41-4427-bdfc-ec0d1b600b64 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:11.619978 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea371e4-4a41-4427-bdfc-ec0d1b600b64 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:11.628367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711c842-8fe5-4fcd-8962-60f5515b813f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:11.629441 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711c842-8fe5-4fcd-8962-60f5515b813f admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:11.629915 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711c842-8fe5-4fcd-8962-60f5515b813f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:11.630724 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711c842-8fe5-4fcd-8962-60f5515b813f admin admin] PATH_INFO: `/v3/projects/b6db73b944a34c859f22a0ecfc4f576d/users/50bd8a8f517346879d98459d7498b557/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:11.667062 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711c842-8fe5-4fcd-8962-60f5515b813f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6db73b944a34c859f22a0ecfc4f576d, user_id=50bd8a8f517346879d98459d7498b557, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:11.671762 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711c842-8fe5-4fcd-8962-60f5515b813f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:11.691344 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:11.718213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5bab8-8efe-4c86-8d1f-6081b0c39327 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:11.719306 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 877/1757] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:11 2022] PUT /identity/v3/projects/b6db73b944a34c859f22a0ecfc4f576d/users/50bd8a8f517346879d98459d7498b557/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:24:11.732960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5bab8-8efe-4c86-8d1f-6081b0c39327 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:11.847214 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2e4c8-7a5c-4a35-b8f8-bbf1935cd1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:11.847968 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2e4c8-7a5c-4a35-b8f8-bbf1935cd1f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:11.848256 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2e4c8-7a5c-4a35-b8f8-bbf1935cd1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:11.848708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2e4c8-7a5c-4a35-b8f8-bbf1935cd1f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:11.849886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2e4c8-7a5c-4a35-b8f8-bbf1935cd1f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:11.855270 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2e4c8-7a5c-4a35-b8f8-bbf1935cd1f6 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:11.875886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 881/1758] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:11.880852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711c842-8fe5-4fcd-8962-60f5515b813f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:11.906359 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711c842-8fe5-4fcd-8962-60f5515b813f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:11.956057 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47703a-122a-49dd-9fb2-549663812548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:12.012476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47703a-122a-49dd-9fb2-549663812548 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:12.013128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47703a-122a-49dd-9fb2-549663812548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:12.013128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47703a-122a-49dd-9fb2-549663812548 admin admin] PATH_INFO: `/v3/projects/b6db73b944a34c859f22a0ecfc4f576d/users/50bd8a8f517346879d98459d7498b557/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:12.025763 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47703a-122a-49dd-9fb2-549663812548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6db73b944a34c859f22a0ecfc4f576d, user_id=50bd8a8f517346879d98459d7498b557, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:12.029727 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47703a-122a-49dd-9fb2-549663812548 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:12.059965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 878/1759] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:11 2022] PUT /identity/v3/projects/b6db73b944a34c859f22a0ecfc4f576d/users/50bd8a8f517346879d98459d7498b557/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:12.061956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c8ffe-3394-4583-a995-6dc002e08926 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:12.062330 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c8ffe-3394-4583-a995-6dc002e08926 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:12.064706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c8ffe-3394-4583-a995-6dc002e08926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:12.170111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-1e5c8ffe-3394-4583-a995-6dc002e08926 None None] MFA Rules not processed for user `50bd8a8f517346879d98459d7498b557`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:12.175430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1e5c8ffe-3394-4583-a995-6dc002e08926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:12.490368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 882/1760] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:12.502923 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47703a-122a-49dd-9fb2-549663812548 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:12.504973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8c47703a-122a-49dd-9fb2-549663812548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:12.628212 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47703a-122a-49dd-9fb2-549663812548 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:12.641255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc67f9-1726-45aa-b920-26d4f2c78a07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:12.644485 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc67f9-1726-45aa-b920-26d4f2c78a07 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:12.645326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc67f9-1726-45aa-b920-26d4f2c78a07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:12.646574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc67f9-1726-45aa-b920-26d4f2c78a07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:12.675545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc67f9-1726-45aa-b920-26d4f2c78a07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:12.677233 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cc67f9-1726-45aa-b920-26d4f2c78a07 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:12.707669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 879/1761] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:12.985603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c8ffe-3394-4583-a995-6dc002e08926 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:13.012219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c8ffe-3394-4583-a995-6dc002e08926 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:13.022311 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7e9e0-4355-4115-8740-ba7f01856eb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:13.023114 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7e9e0-4355-4115-8740-ba7f01856eb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:13.023285 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7e9e0-4355-4115-8740-ba7f01856eb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:13.023734 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7e9e0-4355-4115-8740-ba7f01856eb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:13.199476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7e9e0-4355-4115-8740-ba7f01856eb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:13.204645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7e9e0-4355-4115-8740-ba7f01856eb0 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:13.229688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 883/1762] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:13.612678 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60e140-86d6-4220-914e-443da115a957 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:13.613279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60e140-86d6-4220-914e-443da115a957 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:13.613608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60e140-86d6-4220-914e-443da115a957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:13.696814 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-fd60e140-86d6-4220-914e-443da115a957 None None] MFA Rules not processed for user `5ef6577b10f7426a925d594e627f27cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:13.698553 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-fd60e140-86d6-4220-914e-443da115a957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:13.812757 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f2da3-a7b9-4b40-9790-2841d853a6ec None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:13.813351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f2da3-a7b9-4b40-9790-2841d853a6ec None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:13.813572 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f2da3-a7b9-4b40-9790-2841d853a6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:13.877201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-614f2da3-a7b9-4b40-9790-2841d853a6ec None None] MFA Rules not processed for user `d79f95d9699b4b0393e14a9212986ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:13.886695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-614f2da3-a7b9-4b40-9790-2841d853a6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:13.968159 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:13.993109 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 880/1763] 10.222.0.58 () {58 vars in 1015 bytes} [Mon Aug 1 15:24:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:14.160143 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 884/1764] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:14.264405 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60e140-86d6-4220-914e-443da115a957 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:14.373542 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f2da3-a7b9-4b40-9790-2841d853a6ec None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:14.375463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-614f2da3-a7b9-4b40-9790-2841d853a6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:14.415318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f2da3-a7b9-4b40-9790-2841d853a6ec None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:14.425354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190b1f5-f72b-4545-929c-1ace26fa6f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:14.426287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190b1f5-f72b-4545-929c-1ace26fa6f80 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:14.426840 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190b1f5-f72b-4545-929c-1ace26fa6f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:14.427232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190b1f5-f72b-4545-929c-1ace26fa6f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:14.441748 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190b1f5-f72b-4545-929c-1ace26fa6f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:14.443516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190b1f5-f72b-4545-929c-1ace26fa6f80 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:14.465686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 885/1765] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:14.575371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60e140-86d6-4220-914e-443da115a957 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:14.583772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d62101-6e33-48f5-a94c-16963f6ca176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:14.584634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d62101-6e33-48f5-a94c-16963f6ca176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:14.585153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d62101-6e33-48f5-a94c-16963f6ca176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:14.585484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d62101-6e33-48f5-a94c-16963f6ca176 admin admin] PATH_INFO: `/v3/users/fb4d21e643ef4f78bc471461c76936f1` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:14.587112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d62101-6e33-48f5-a94c-16963f6ca176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb4d21e643ef4f78bc471461c76936f1)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:14.592328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d62101-6e33-48f5-a94c-16963f6ca176 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:14.703948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:14.732307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-37d62101-6e33-48f5-a94c-16963f6ca176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb4d21e643ef4f78bc471461c76936f1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:14.734209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-37d62101-6e33-48f5-a94c-16963f6ca176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb4d21e643ef4f78bc471461c76936f1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:14.802257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-37d62101-6e33-48f5-a94c-16963f6ca176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb4d21e643ef4f78bc471461c76936f1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:14.835905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-37d62101-6e33-48f5-a94c-16963f6ca176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb4d21e643ef4f78bc471461c76936f1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:14.864605 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-37d62101-6e33-48f5-a94c-16963f6ca176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb4d21e643ef4f78bc471461c76936f1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:14.900445 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 881/1766] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:14 2022] DELETE /identity/v3/users/fb4d21e643ef4f78bc471461c76936f1 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:14.904253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546f1db-154d-420a-ac50-79106ac08bba None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:14.906320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546f1db-154d-420a-ac50-79106ac08bba None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:14.906884 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7546f1db-154d-420a-ac50-79106ac08bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:14.956570 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-7546f1db-154d-420a-ac50-79106ac08bba None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:14.958418 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-7546f1db-154d-420a-ac50-79106ac08bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:15.103468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 886/1767] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:15.263154 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d62101-6e33-48f5-a94c-16963f6ca176 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:15.265012 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-37d62101-6e33-48f5-a94c-16963f6ca176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:15.396795 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d62101-6e33-48f5-a94c-16963f6ca176 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:15.408406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc0e2b-f3e5-4a3c-a4ee-3d5cc5068a7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:15.409089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc0e2b-f3e5-4a3c-a4ee-3d5cc5068a7e service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:15.409368 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc0e2b-f3e5-4a3c-a4ee-3d5cc5068a7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:15.409642 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc0e2b-f3e5-4a3c-a4ee-3d5cc5068a7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:15.430975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc0e2b-f3e5-4a3c-a4ee-3d5cc5068a7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:15.433314 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc0e2b-f3e5-4a3c-a4ee-3d5cc5068a7e service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:15.476144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 882/1768] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:16.621173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c44557-db09-4574-b299-4da22cc35451 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:16.622125 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c44557-db09-4574-b299-4da22cc35451 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:16.622554 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c44557-db09-4574-b299-4da22cc35451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:16.705330 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-38c44557-db09-4574-b299-4da22cc35451 None None] MFA Rules not processed for user `93d86ae4501e43d59d172d2e7acfdd54`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:16.706615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-38c44557-db09-4574-b299-4da22cc35451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:16.949760 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 887/1769] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:18.710780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc0e2b-f3e5-4a3c-a4ee-3d5cc5068a7e service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:18.712460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8dbc0e2b-f3e5-4a3c-a4ee-3d5cc5068a7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:18.761885 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc0e2b-f3e5-4a3c-a4ee-3d5cc5068a7e service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:18.790123 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e98f7-ea04-4c44-89a0-36ff10016abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:18.791050 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e98f7-ea04-4c44-89a0-36ff10016abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:18.791504 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e98f7-ea04-4c44-89a0-36ff10016abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:18.791950 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e98f7-ea04-4c44-89a0-36ff10016abf admin admin] PATH_INFO: `/v3/users/3a5c31b36e5249e890db4ed180faaa71` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:18.793483 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e98f7-ea04-4c44-89a0-36ff10016abf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a5c31b36e5249e890db4ed180faaa71)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:18.796103 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e98f7-ea04-4c44-89a0-36ff10016abf admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:18.961846 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-312e98f7-ea04-4c44-89a0-36ff10016abf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a5c31b36e5249e890db4ed180faaa71'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:18.963094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-312e98f7-ea04-4c44-89a0-36ff10016abf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a5c31b36e5249e890db4ed180faaa71'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:18.988356 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-312e98f7-ea04-4c44-89a0-36ff10016abf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a5c31b36e5249e890db4ed180faaa71'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:19.007499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-312e98f7-ea04-4c44-89a0-36ff10016abf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a5c31b36e5249e890db4ed180faaa71'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:19.043555 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-312e98f7-ea04-4c44-89a0-36ff10016abf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a5c31b36e5249e890db4ed180faaa71'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:19.072096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 883/1770] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:18 2022] DELETE /identity/v3/users/3a5c31b36e5249e890db4ed180faaa71 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:19.075259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b45adf-33d4-4749-af9c-d3efeebd10c9 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:19.075896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b45adf-33d4-4749-af9c-d3efeebd10c9 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:19.076197 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b45adf-33d4-4749-af9c-d3efeebd10c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:19.159035 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-f1b45adf-33d4-4749-af9c-d3efeebd10c9 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:19.160735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f1b45adf-33d4-4749-af9c-d3efeebd10c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:19.295076 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e98f7-ea04-4c44-89a0-36ff10016abf admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:19.297005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-312e98f7-ea04-4c44-89a0-36ff10016abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:19.341924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:19.393237 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 888/1771] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:19.426584 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:19.518671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b45adf-33d4-4749-af9c-d3efeebd10c9 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:19.520126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f1b45adf-33d4-4749-af9c-d3efeebd10c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:19.540797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e98f7-ea04-4c44-89a0-36ff10016abf admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:19.573013 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92097cf1-cb91-4408-8ced-b620ed7e3ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:19.573824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92097cf1-cb91-4408-8ced-b620ed7e3ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:19.574100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92097cf1-cb91-4408-8ced-b620ed7e3ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:19.574408 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92097cf1-cb91-4408-8ced-b620ed7e3ced admin admin] PATH_INFO: `/v3/projects/4c922afd0c304da0b219ad0265c3ee6a` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:19.585564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92097cf1-cb91-4408-8ced-b620ed7e3ced admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c922afd0c304da0b219ad0265c3ee6a)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:19.604896 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92097cf1-cb91-4408-8ced-b620ed7e3ced admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:19.775872 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-92097cf1-cb91-4408-8ced-b620ed7e3ced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c922afd0c304da0b219ad0265c3ee6a', 'request_id': 'req-92097cf1-cb91-4408-8ced-b620ed7e3ced'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:19.792474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-92097cf1-cb91-4408-8ced-b620ed7e3ced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c922afd0c304da0b219ad0265c3ee6a', 'request_id': 'req-92097cf1-cb91-4408-8ced-b620ed7e3ced'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:19.817625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-92097cf1-cb91-4408-8ced-b620ed7e3ced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c922afd0c304da0b219ad0265c3ee6a', 'request_id': 'req-92097cf1-cb91-4408-8ced-b620ed7e3ced'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:19.823582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b45adf-33d4-4749-af9c-d3efeebd10c9 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:19.845751 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-92097cf1-cb91-4408-8ced-b620ed7e3ced admin admin] The token cache is being invalidate because project 4c922afd0c304da0b219ad0265c3ee6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:19.846175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-92097cf1-cb91-4408-8ced-b620ed7e3ced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:19.847082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-92097cf1-cb91-4408-8ced-b620ed7e3ced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:19.850661 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 884/1772] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:19 2022] DELETE /identity/v3/projects/4c922afd0c304da0b219ad0265c3ee6a => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:19.854752 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92097cf1-cb91-4408-8ced-b620ed7e3ced admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:19.862970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-92097cf1-cb91-4408-8ced-b620ed7e3ced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:20.081561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e709d-87ec-4846-83e1-33999a51d3cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:20.082431 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e709d-87ec-4846-83e1-33999a51d3cc service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:20.082807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e709d-87ec-4846-83e1-33999a51d3cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:20.083183 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e709d-87ec-4846-83e1-33999a51d3cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:20.090279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8b9e709d-87ec-4846-83e1-33999a51d3cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:20.111815 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92097cf1-cb91-4408-8ced-b620ed7e3ced admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:20.149672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca96eb-b130-424c-960e-5660f53fbdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:20.151426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca96eb-b130-424c-960e-5660f53fbdc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:20.156847 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e709d-87ec-4846-83e1-33999a51d3cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:20.157671 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca96eb-b130-424c-960e-5660f53fbdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:20.158234 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca96eb-b130-424c-960e-5660f53fbdc4 admin admin] PATH_INFO: `/v3/users/50bd8a8f517346879d98459d7498b557` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:20.161470 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e709d-87ec-4846-83e1-33999a51d3cc service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:20.182735 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca96eb-b130-424c-960e-5660f53fbdc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50bd8a8f517346879d98459d7498b557)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:20.188503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca96eb-b130-424c-960e-5660f53fbdc4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:20.223600 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 889/1773] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 705 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:24:20.367356 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-fcca96eb-b130-424c-960e-5660f53fbdc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50bd8a8f517346879d98459d7498b557'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:20.368259 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-fcca96eb-b130-424c-960e-5660f53fbdc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50bd8a8f517346879d98459d7498b557'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:20.411899 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:20.416187 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-fcca96eb-b130-424c-960e-5660f53fbdc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50bd8a8f517346879d98459d7498b557'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:20.435954 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-fcca96eb-b130-424c-960e-5660f53fbdc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50bd8a8f517346879d98459d7498b557'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:20.454596 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-fcca96eb-b130-424c-960e-5660f53fbdc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50bd8a8f517346879d98459d7498b557'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:20.487929 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 885/1774] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:19 2022] DELETE /identity/v3/users/50bd8a8f517346879d98459d7498b557 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:24:21.521808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529687d-0d77-4093-bbe1-fd7ef0969e44 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:21.522413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529687d-0d77-4093-bbe1-fd7ef0969e44 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:21.522413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529687d-0d77-4093-bbe1-fd7ef0969e44 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:21.524113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 890/1775] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 15:24:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:24:21.560830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f08634-6547-4e67-83ab-1c9d7e9a3329 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:21.562232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f08634-6547-4e67-83ab-1c9d7e9a3329 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:21.562714 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f08634-6547-4e67-83ab-1c9d7e9a3329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:21.627548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-32f08634-6547-4e67-83ab-1c9d7e9a3329 None None] MFA Rules not processed for user `2de0ff99a2b74979a28a8b10e9ebdfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:21.631175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-32f08634-6547-4e67-83ab-1c9d7e9a3329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:21.898967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 886/1776] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 15:24:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:23.416234 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529687d-0d77-4093-bbe1-fd7ef0969e44 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:23.420726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-b529687d-0d77-4093-bbe1-fd7ef0969e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:23.694108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529687d-0d77-4093-bbe1-fd7ef0969e44 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:23.706893 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:23.707354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:23.707846 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:23.708346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f admin admin] PATH_INFO: `/v3/projects/b6db73b944a34c859f22a0ecfc4f576d` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:23.710585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6db73b944a34c859f22a0ecfc4f576d)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:23.724615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:23.867336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6db73b944a34c859f22a0ecfc4f576d', 'request_id': 'req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:23.872539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:23.898985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6db73b944a34c859f22a0ecfc4f576d', 'request_id': 'req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:23.914121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6db73b944a34c859f22a0ecfc4f576d', 'request_id': 'req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:23.978771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f admin admin] The token cache is being invalidate because project b6db73b944a34c859f22a0ecfc4f576d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:23.990095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:23.991615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:23.995886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 891/1777] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:23 2022] DELETE /identity/v3/projects/b6db73b944a34c859f22a0ecfc4f576d => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:24.000765 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977c570-92f4-4acb-b564-5bf0496d8b5b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:24.001331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977c570-92f4-4acb-b564-5bf0496d8b5b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:24.001412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1977c570-92f4-4acb-b564-5bf0496d8b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:24.055960 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-1977c570-92f4-4acb-b564-5bf0496d8b5b None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:24.057805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1977c570-92f4-4acb-b564-5bf0496d8b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:24.217944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:24.220055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:24.333569 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 887/1778] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:24:24.344128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c7343-5858-4258-9839-167236d96c48 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:24.344647 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c7343-5858-4258-9839-167236d96c48 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:24.346328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c7343-5858-4258-9839-167236d96c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:24.409870 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-316c7343-5858-4258-9839-167236d96c48 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:24.412939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-316c7343-5858-4258-9839-167236d96c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:24.418394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f9fcb-e9a0-4e05-bd79-0162ce68bc6f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:24.432194 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:24.433923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:24.434600 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:24.435120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b admin admin] PATH_INFO: `/v3/projects/49cbcc6897e2410588125aba7c6fbb85` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:24.440840 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b admin admin] RBAC: Authorizing `identity:delete_project(project_id=49cbcc6897e2410588125aba7c6fbb85)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:24.444819 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:24.447866 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 888/1779] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:24.449552 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c7343-5858-4258-9839-167236d96c48 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:24.464105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c7343-5858-4258-9839-167236d96c48 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:24.472043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d5669-65ab-458b-81a8-dd0a9691e778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:24.473217 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d5669-65ab-458b-81a8-dd0a9691e778 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:24.473850 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d5669-65ab-458b-81a8-dd0a9691e778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:24.474317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d5669-65ab-458b-81a8-dd0a9691e778 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:24.475577 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d5669-65ab-458b-81a8-dd0a9691e778 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:24:24.476063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d5669-65ab-458b-81a8-dd0a9691e778 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:24.478380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d5669-65ab-458b-81a8-dd0a9691e778 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:24.503277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 889/1780] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:24:24.504367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d5669-65ab-458b-81a8-dd0a9691e778 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:24.532572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d5669-65ab-458b-81a8-dd0a9691e778 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:24.540695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac54f00-09f5-4caf-9e8f-e44ae79f6033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:24.541654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac54f00-09f5-4caf-9e8f-e44ae79f6033 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:24.542750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac54f00-09f5-4caf-9e8f-e44ae79f6033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:24.543347 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac54f00-09f5-4caf-9e8f-e44ae79f6033 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:24.544716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac54f00-09f5-4caf-9e8f-e44ae79f6033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:24.547297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac54f00-09f5-4caf-9e8f-e44ae79f6033 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:24.576089 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49cbcc6897e2410588125aba7c6fbb85', 'request_id': 'req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:24.594272 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49cbcc6897e2410588125aba7c6fbb85', 'request_id': 'req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:24.605148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49cbcc6897e2410588125aba7c6fbb85', 'request_id': 'req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:24.625042 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 890/1781] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:24 2022] POST /identity/v3/projects => generated 361 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:24.629047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac54f00-09f5-4caf-9e8f-e44ae79f6033 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:24.659773 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b admin admin] The token cache is being invalidate because project 49cbcc6897e2410588125aba7c6fbb85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:24.660094 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:24.668347 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:24.671091 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 892/1782] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:24 2022] DELETE /identity/v3/projects/49cbcc6897e2410588125aba7c6fbb85 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:24:24.675393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:24.677037 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:24.692622 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:24.782484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac54f00-09f5-4caf-9e8f-e44ae79f6033 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:24.809072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e62067-f902-45be-ab8c-57b4bc179ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:24.813337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e62067-f902-45be-ab8c-57b4bc179ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:24.815480 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e62067-f902-45be-ab8c-57b4bc179ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:24.816403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e62067-f902-45be-ab8c-57b4bc179ca4 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:24.824503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e62067-f902-45be-ab8c-57b4bc179ca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:24.835300 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e62067-f902-45be-ab8c-57b4bc179ca4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:24.911403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 891/1783] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:24 2022] POST /identity/v3/users => generated 281 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:24.912242 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa0cb2-ba48-4917-9e61-b20cb3dae00b admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:24.912790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e62067-f902-45be-ab8c-57b4bc179ca4 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:24.914678 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-28e62067-f902-45be-ab8c-57b4bc179ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:24.931530 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6fc94-7ed8-4652-b3c3-ea7d7dbc4538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:24.934574 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6fc94-7ed8-4652-b3c3-ea7d7dbc4538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:24.935588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6fc94-7ed8-4652-b3c3-ea7d7dbc4538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:24.936772 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6fc94-7ed8-4652-b3c3-ea7d7dbc4538 admin admin] PATH_INFO: `/v3/users/4da5835c823c4568bace5b9d7afd6614` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:24.943885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6fc94-7ed8-4652-b3c3-ea7d7dbc4538 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4da5835c823c4568bace5b9d7afd6614)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:24.944169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e62067-f902-45be-ab8c-57b4bc179ca4 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:24.951339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6fc94-7ed8-4652-b3c3-ea7d7dbc4538 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:24.960933 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a9a88-be52-4f72-b0a1-8c175cf303de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:24.961729 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a9a88-be52-4f72-b0a1-8c175cf303de admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:24.962191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a9a88-be52-4f72-b0a1-8c175cf303de admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:24.962471 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a9a88-be52-4f72-b0a1-8c175cf303de admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:24.963473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a9a88-be52-4f72-b0a1-8c175cf303de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:24.965700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6a9a88-be52-4f72-b0a1-8c175cf303de admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:24.999927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 892/1784] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:25.008075 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a9a88-be52-4f72-b0a1-8c175cf303de admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:25.011393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:25.030371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a9a88-be52-4f72-b0a1-8c175cf303de admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:25.045505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0bba0-a426-4098-a3d1-29b175a419e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:25.046203 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0bba0-a426-4098-a3d1-29b175a419e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:25.046479 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0bba0-a426-4098-a3d1-29b175a419e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:25.046738 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0bba0-a426-4098-a3d1-29b175a419e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:25.047448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0bba0-a426-4098-a3d1-29b175a419e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:25.049105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0bba0-a426-4098-a3d1-29b175a419e2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:25.067564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 893/1785] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:25.068612 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0bba0-a426-4098-a3d1-29b175a419e2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:25.085844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0bba0-a426-4098-a3d1-29b175a419e2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:25.092407 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de1239-ac3a-4e96-95e0-7683ac208a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:25.093124 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de1239-ac3a-4e96-95e0-7683ac208a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:25.093552 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de1239-ac3a-4e96-95e0-7683ac208a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:25.093903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de1239-ac3a-4e96-95e0-7683ac208a6f admin admin] PATH_INFO: `/v3/projects/2629d153c4ca40e1ba715a9fc7599e1d/users/c6ffacec26bf48cfa5dbc458a7bcec81/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:25.095496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-e1f6fc94-7ed8-4652-b3c3-ea7d7dbc4538 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4da5835c823c4568bace5b9d7afd6614'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:25.112544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-e1f6fc94-7ed8-4652-b3c3-ea7d7dbc4538 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4da5835c823c4568bace5b9d7afd6614'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:25.113511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-e1f6fc94-7ed8-4652-b3c3-ea7d7dbc4538 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4da5835c823c4568bace5b9d7afd6614'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:25.121414 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de1239-ac3a-4e96-95e0-7683ac208a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2629d153c4ca40e1ba715a9fc7599e1d, user_id=c6ffacec26bf48cfa5dbc458a7bcec81, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:25.124476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de1239-ac3a-4e96-95e0-7683ac208a6f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:25.129389 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-e1f6fc94-7ed8-4652-b3c3-ea7d7dbc4538 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4da5835c823c4568bace5b9d7afd6614'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:25.154881 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 894/1786] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:25 2022] PUT /identity/v3/projects/2629d153c4ca40e1ba715a9fc7599e1d/users/c6ffacec26bf48cfa5dbc458a7bcec81/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:25.155510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de1239-ac3a-4e96-95e0-7683ac208a6f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:25.158990 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-e1f6fc94-7ed8-4652-b3c3-ea7d7dbc4538 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4da5835c823c4568bace5b9d7afd6614'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:25.166006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b5de1239-ac3a-4e96-95e0-7683ac208a6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:25.167887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 893/1787] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:24 2022] DELETE /identity/v3/users/4da5835c823c4568bace5b9d7afd6614 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:25.173715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd0471-03b7-488a-aeb2-f788301ffdf8 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:25.173715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd0471-03b7-488a-aeb2-f788301ffdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:25.174123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd0471-03b7-488a-aeb2-f788301ffdf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:25.228984 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-51cd0471-03b7-488a-aeb2-f788301ffdf8 None None] MFA Rules not processed for user `c6ffacec26bf48cfa5dbc458a7bcec81`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:25.231807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-51cd0471-03b7-488a-aeb2-f788301ffdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:25.344051 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de1239-ac3a-4e96-95e0-7683ac208a6f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:25.404263 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 894/1788] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:25.450983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd0471-03b7-488a-aeb2-f788301ffdf8 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:25.453561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-51cd0471-03b7-488a-aeb2-f788301ffdf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:25.524702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79712635-97dd-4722-b1f2-662abcf877ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:25.525560 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712635-97dd-4722-b1f2-662abcf877ba service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:25.526999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712635-97dd-4722-b1f2-662abcf877ba service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:25.527440 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712635-97dd-4722-b1f2-662abcf877ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:25.535707 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-79712635-97dd-4722-b1f2-662abcf877ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:25.564430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79712635-97dd-4722-b1f2-662abcf877ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:25.565346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd0471-03b7-488a-aeb2-f788301ffdf8 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:25.566192 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79712635-97dd-4722-b1f2-662abcf877ba service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:25.569843 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea23889-f6fe-42bf-8977-329f2b948866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:25.571032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea23889-f6fe-42bf-8977-329f2b948866 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:25.571493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea23889-f6fe-42bf-8977-329f2b948866 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:25.571802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:25.572194 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea23889-f6fe-42bf-8977-329f2b948866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:25.583274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 895/1789] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:25.584769 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea23889-f6fe-42bf-8977-329f2b948866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:25.586327 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea23889-f6fe-42bf-8977-329f2b948866 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:25.598092 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 895/1790] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:25.613123 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79712635-97dd-4722-b1f2-662abcf877ba service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:25.616335 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-79712635-97dd-4722-b1f2-662abcf877ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:25.704479 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79712635-97dd-4722-b1f2-662abcf877ba service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:25.711490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9758e0-d5be-4f77-a28b-d2097a9f709c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:25.712196 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9758e0-d5be-4f77-a28b-d2097a9f709c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:25.713180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9758e0-d5be-4f77-a28b-d2097a9f709c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:25.713180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9758e0-d5be-4f77-a28b-d2097a9f709c admin admin] PATH_INFO: `/v3/users/c6ffacec26bf48cfa5dbc458a7bcec81` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:25.714980 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9758e0-d5be-4f77-a28b-d2097a9f709c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6ffacec26bf48cfa5dbc458a7bcec81)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:25.718298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9758e0-d5be-4f77-a28b-d2097a9f709c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:25.823299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-da9758e0-d5be-4f77-a28b-d2097a9f709c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6ffacec26bf48cfa5dbc458a7bcec81'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:25.824238 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-da9758e0-d5be-4f77-a28b-d2097a9f709c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6ffacec26bf48cfa5dbc458a7bcec81'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:25.850471 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-da9758e0-d5be-4f77-a28b-d2097a9f709c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6ffacec26bf48cfa5dbc458a7bcec81'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:25.860579 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-da9758e0-d5be-4f77-a28b-d2097a9f709c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6ffacec26bf48cfa5dbc458a7bcec81'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:25.870239 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-da9758e0-d5be-4f77-a28b-d2097a9f709c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6ffacec26bf48cfa5dbc458a7bcec81'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:25.881859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 896/1791] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:25 2022] DELETE /identity/v3/users/c6ffacec26bf48cfa5dbc458a7bcec81 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:25.885426 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a609211-5880-4ede-be04-ee69a380b68b None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:25.886254 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a609211-5880-4ede-be04-ee69a380b68b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:25.886535 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a609211-5880-4ede-be04-ee69a380b68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:25.915746 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-4a609211-5880-4ede-be04-ee69a380b68b None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:25.917224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4a609211-5880-4ede-be04-ee69a380b68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:25.949667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:25.997819 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 896/1792] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:26.012185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9758e0-d5be-4f77-a28b-d2097a9f709c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:26.014665 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-da9758e0-d5be-4f77-a28b-d2097a9f709c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:26.170653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9758e0-d5be-4f77-a28b-d2097a9f709c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:26.175609 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3e3e3-3735-421e-b4f1-3eb3c761a5b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:26.176574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3e3e3-3735-421e-b4f1-3eb3c761a5b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:26.177051 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3e3e3-3735-421e-b4f1-3eb3c761a5b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:26.177630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3e3e3-3735-421e-b4f1-3eb3c761a5b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:26.180981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b905b-0450-4de7-b547-15bf678d3bcb None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:26.180981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b905b-0450-4de7-b547-15bf678d3bcb None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:26.180981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b905b-0450-4de7-b547-15bf678d3bcb None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:26.184434 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 897/1793] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 15:24:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:24:26.193938 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea3e3e3-3735-421e-b4f1-3eb3c761a5b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:26.195831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea3e3e3-3735-421e-b4f1-3eb3c761a5b5 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:26.209817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 897/1794] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:24:26.226782 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1bd15-4790-4f51-9544-6a1b26a7a210 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:26.227145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1bd15-4790-4f51-9544-6a1b26a7a210 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:26.227250 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1bd15-4790-4f51-9544-6a1b26a7a210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:26.257121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-cfa1bd15-4790-4f51-9544-6a1b26a7a210 None None] MFA Rules not processed for user `2de0ff99a2b74979a28a8b10e9ebdfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:26.258424 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-cfa1bd15-4790-4f51-9544-6a1b26a7a210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:26.380790 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 898/1795] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 15:24:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:24:26.616700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3e3e3-3735-421e-b4f1-3eb3c761a5b5 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:26.618585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8ea3e3e3-3735-421e-b4f1-3eb3c761a5b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:26.632257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3e3e3-3735-421e-b4f1-3eb3c761a5b5 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:26.639367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21398bc5-02c4-4a2c-9e0f-471266465534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:26.640195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21398bc5-02c4-4a2c-9e0f-471266465534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:26.640499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21398bc5-02c4-4a2c-9e0f-471266465534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:26.640937 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21398bc5-02c4-4a2c-9e0f-471266465534 admin admin] PATH_INFO: `/v3/projects/2629d153c4ca40e1ba715a9fc7599e1d` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:26.642469 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21398bc5-02c4-4a2c-9e0f-471266465534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2629d153c4ca40e1ba715a9fc7599e1d)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:26.644756 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21398bc5-02c4-4a2c-9e0f-471266465534 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:26.666831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:26.708128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-21398bc5-02c4-4a2c-9e0f-471266465534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2629d153c4ca40e1ba715a9fc7599e1d', 'request_id': 'req-21398bc5-02c4-4a2c-9e0f-471266465534'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:26.720699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-21398bc5-02c4-4a2c-9e0f-471266465534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2629d153c4ca40e1ba715a9fc7599e1d', 'request_id': 'req-21398bc5-02c4-4a2c-9e0f-471266465534'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:26.729843 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-21398bc5-02c4-4a2c-9e0f-471266465534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2629d153c4ca40e1ba715a9fc7599e1d', 'request_id': 'req-21398bc5-02c4-4a2c-9e0f-471266465534'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:26.744873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-21398bc5-02c4-4a2c-9e0f-471266465534 admin admin] The token cache is being invalidate because project 2629d153c4ca40e1ba715a9fc7599e1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:26.745184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-21398bc5-02c4-4a2c-9e0f-471266465534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:26.745890 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-21398bc5-02c4-4a2c-9e0f-471266465534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:26.748872 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 898/1796] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:26 2022] DELETE /identity/v3/projects/2629d153c4ca40e1ba715a9fc7599e1d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:26.754347 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53583e2-e56c-4332-9dfe-9654379b328c None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:26.754759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53583e2-e56c-4332-9dfe-9654379b328c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:26.755516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53583e2-e56c-4332-9dfe-9654379b328c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:26.791057 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21398bc5-02c4-4a2c-9e0f-471266465534 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:26.795442 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-21398bc5-02c4-4a2c-9e0f-471266465534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:26.800327 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-f53583e2-e56c-4332-9dfe-9654379b328c None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:26.807628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f53583e2-e56c-4332-9dfe-9654379b328c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:26.907040 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 899/1797] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:26.916087 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe4550-d57d-46d1-9dd7-0045584c789d None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:26.916396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe4550-d57d-46d1-9dd7-0045584c789d None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:26.916632 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe4550-d57d-46d1-9dd7-0045584c789d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:26.942376 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-3cbe4550-d57d-46d1-9dd7-0045584c789d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:26.943996 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-3cbe4550-d57d-46d1-9dd7-0045584c789d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:26.957507 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 900/1798] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:26.959543 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe4550-d57d-46d1-9dd7-0045584c789d None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:26.964009 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21398bc5-02c4-4a2c-9e0f-471266465534 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:26.971298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe4550-d57d-46d1-9dd7-0045584c789d None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:26.972457 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131c85-c83d-487b-89bf-b538c9ac2593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:26.973684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131c85-c83d-487b-89bf-b538c9ac2593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:26.974314 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131c85-c83d-487b-89bf-b538c9ac2593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:26.975883 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131c85-c83d-487b-89bf-b538c9ac2593 admin admin] PATH_INFO: `/v3/projects/729155ca3de245a08b6934dc7119d334` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:26.977981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b39f2f-cc70-41ee-a6dd-75b05d9b8a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:26.978135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82131c85-c83d-487b-89bf-b538c9ac2593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=729155ca3de245a08b6934dc7119d334)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:26.978913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b39f2f-cc70-41ee-a6dd-75b05d9b8a53 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:26.979196 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b39f2f-cc70-41ee-a6dd-75b05d9b8a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:26.979476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b39f2f-cc70-41ee-a6dd-75b05d9b8a53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:26.980182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b39f2f-cc70-41ee-a6dd-75b05d9b8a53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:24:26.980490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b39f2f-cc70-41ee-a6dd-75b05d9b8a53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:26.982331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b39f2f-cc70-41ee-a6dd-75b05d9b8a53 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:26.982896 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82131c85-c83d-487b-89bf-b538c9ac2593 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:27.000572 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 901/1799] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:24:27.001616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b39f2f-cc70-41ee-a6dd-75b05d9b8a53 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:27.014537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b39f2f-cc70-41ee-a6dd-75b05d9b8a53 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:27.023119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9fef64-99db-41f6-bc73-fb4d95568132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:27.023793 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fef64-99db-41f6-bc73-fb4d95568132 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:27.024057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fef64-99db-41f6-bc73-fb4d95568132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:27.024399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fef64-99db-41f6-bc73-fb4d95568132 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:27.025076 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9fef64-99db-41f6-bc73-fb4d95568132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:27.027027 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9fef64-99db-41f6-bc73-fb4d95568132 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:27.071545 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 902/1800] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:27 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:27.073062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9fef64-99db-41f6-bc73-fb4d95568132 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:27.085109 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82131c85-c83d-487b-89bf-b538c9ac2593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '729155ca3de245a08b6934dc7119d334', 'request_id': 'req-82131c85-c83d-487b-89bf-b538c9ac2593'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:27.089016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-ae9fef64-99db-41f6-bc73-fb4d95568132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:27.105530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82131c85-c83d-487b-89bf-b538c9ac2593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '729155ca3de245a08b6934dc7119d334', 'request_id': 'req-82131c85-c83d-487b-89bf-b538c9ac2593'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:27.133080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:27.139721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82131c85-c83d-487b-89bf-b538c9ac2593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '729155ca3de245a08b6934dc7119d334', 'request_id': 'req-82131c85-c83d-487b-89bf-b538c9ac2593'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:27.168198 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82131c85-c83d-487b-89bf-b538c9ac2593 admin admin] The token cache is being invalidate because project 729155ca3de245a08b6934dc7119d334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:27.168805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82131c85-c83d-487b-89bf-b538c9ac2593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:27.170119 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-82131c85-c83d-487b-89bf-b538c9ac2593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:27.173166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 899/1801] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:26 2022] DELETE /identity/v3/projects/729155ca3de245a08b6934dc7119d334 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:27.176696 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131c85-c83d-487b-89bf-b538c9ac2593 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:27.183091 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-82131c85-c83d-487b-89bf-b538c9ac2593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:27.223271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9fef64-99db-41f6-bc73-fb4d95568132 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:27.225702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:27.262730 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131c85-c83d-487b-89bf-b538c9ac2593 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:27.268175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae361dda-dba5-4199-830c-783a62c54ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:27.268750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae361dda-dba5-4199-830c-783a62c54ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:27.269262 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae361dda-dba5-4199-830c-783a62c54ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:27.269635 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae361dda-dba5-4199-830c-783a62c54ed2 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:27.270395 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae361dda-dba5-4199-830c-783a62c54ed2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:27.272612 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae361dda-dba5-4199-830c-783a62c54ed2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:27.333619 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 900/1802] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:27 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:27.333619 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34162e9b-ce29-4259-8930-95333c7e7c1d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:27.333619 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34162e9b-ce29-4259-8930-95333c7e7c1d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:27.333619 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34162e9b-ce29-4259-8930-95333c7e7c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:27.376574 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031af1b4-55cf-4385-8313-0f8b9e7da3d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:27.377809 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031af1b4-55cf-4385-8313-0f8b9e7da3d2 service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:27.378910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031af1b4-55cf-4385-8313-0f8b9e7da3d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:27.379557 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031af1b4-55cf-4385-8313-0f8b9e7da3d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:27.382253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-031af1b4-55cf-4385-8313-0f8b9e7da3d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:27.386989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-34162e9b-ce29-4259-8930-95333c7e7c1d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:27.391286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-34162e9b-ce29-4259-8930-95333c7e7c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:27.402079 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031af1b4-55cf-4385-8313-0f8b9e7da3d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:27.404004 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031af1b4-55cf-4385-8313-0f8b9e7da3d2 service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:27.413675 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 901/1803] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:27.425133 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34162e9b-ce29-4259-8930-95333c7e7c1d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:27.433314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 903/1804] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:27.438599 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881df0e-ae29-4bc2-87c5-cc27de904581 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:27.439174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881df0e-ae29-4bc2-87c5-cc27de904581 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:27.441394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881df0e-ae29-4bc2-87c5-cc27de904581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:27.447681 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34162e9b-ce29-4259-8930-95333c7e7c1d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:27.453917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ec3e4-6238-4477-ac9b-23f343b754ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:27.454579 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ec3e4-6238-4477-ac9b-23f343b754ff admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:27.455803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ec3e4-6238-4477-ac9b-23f343b754ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:27.456141 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ec3e4-6238-4477-ac9b-23f343b754ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:27.456864 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ec3e4-6238-4477-ac9b-23f343b754ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:27.460853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ec3e4-6238-4477-ac9b-23f343b754ff admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:27.474406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 902/1805] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:27.479539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ec3e4-6238-4477-ac9b-23f343b754ff admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:27.492477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ec3e4-6238-4477-ac9b-23f343b754ff admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:27.493072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-9881df0e-ae29-4bc2-87c5-cc27de904581 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:27.494423 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9881df0e-ae29-4bc2-87c5-cc27de904581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:27.518116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 904/1806] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:27.520656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881df0e-ae29-4bc2-87c5-cc27de904581 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:27.525563 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d8877-fcd2-4d3a-a901-1a853fb91cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:27.526451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d8877-fcd2-4d3a-a901-1a853fb91cde admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:27.526839 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d8877-fcd2-4d3a-a901-1a853fb91cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:27.527209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d8877-fcd2-4d3a-a901-1a853fb91cde admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:27.528053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d8877-fcd2-4d3a-a901-1a853fb91cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:27.530455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d8877-fcd2-4d3a-a901-1a853fb91cde admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:27.545285 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881df0e-ae29-4bc2-87c5-cc27de904581 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:27.560904 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c79a67-2988-41c9-8525-201726a59f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:27.561898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 903/1807] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:27.564923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c79a67-2988-41c9-8525-201726a59f60 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:27.565247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c79a67-2988-41c9-8525-201726a59f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:27.565953 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d8877-fcd2-4d3a-a901-1a853fb91cde admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:27.566123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c79a67-2988-41c9-8525-201726a59f60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:27.566833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c79a67-2988-41c9-8525-201726a59f60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:24:27.567359 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c79a67-2988-41c9-8525-201726a59f60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:27.572593 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c79a67-2988-41c9-8525-201726a59f60 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:27.580929 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d8877-fcd2-4d3a-a901-1a853fb91cde admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:27.588190 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8136d-a269-4988-977f-042580e745d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:27.590215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8136d-a269-4988-977f-042580e745d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:27.590589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8136d-a269-4988-977f-042580e745d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:27.592766 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8136d-a269-4988-977f-042580e745d3 admin admin] PATH_INFO: `/v3/projects/a75bb40bbb7540bab94903f318b208ee/users/0e8811f413d1411da8b19630feeb7cfb/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:27.599953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 905/1808] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:24:27.606063 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c79a67-2988-41c9-8525-201726a59f60 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:27.618589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c79a67-2988-41c9-8525-201726a59f60 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:27.624752 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d3e60-cf6a-4336-a7ed-260f525145cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:27.626368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d3e60-cf6a-4336-a7ed-260f525145cb admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:27.627018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d3e60-cf6a-4336-a7ed-260f525145cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:27.627615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d3e60-cf6a-4336-a7ed-260f525145cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:27.628653 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d3e60-cf6a-4336-a7ed-260f525145cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:27.632101 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d3e60-cf6a-4336-a7ed-260f525145cb admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:27.632738 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8136d-a269-4988-977f-042580e745d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a75bb40bbb7540bab94903f318b208ee, user_id=0e8811f413d1411da8b19630feeb7cfb, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:27.645595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8136d-a269-4988-977f-042580e745d3 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:27.675170 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 906/1809] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:27 2022] POST /identity/v3/projects => generated 369 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:27.679539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d3e60-cf6a-4336-a7ed-260f525145cb admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:27.700180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 904/1810] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:27 2022] PUT /identity/v3/projects/a75bb40bbb7540bab94903f318b208ee/users/0e8811f413d1411da8b19630feeb7cfb/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:27.725171 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44defde8-cc1f-42ca-8ec2-e31f7806d236 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:27.725357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44defde8-cc1f-42ca-8ec2-e31f7806d236 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:27.725636 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44defde8-cc1f-42ca-8ec2-e31f7806d236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:27.819027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-44defde8-cc1f-42ca-8ec2-e31f7806d236 None None] MFA Rules not processed for user `0e8811f413d1411da8b19630feeb7cfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:27.822066 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-44defde8-cc1f-42ca-8ec2-e31f7806d236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:27.858415 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:27.869425 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d3e60-cf6a-4336-a7ed-260f525145cb admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:27.888293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8037ee-993c-4cd7-88f8-0a6ae2925601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:27.888955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8037ee-993c-4cd7-88f8-0a6ae2925601 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:27.889235 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8037ee-993c-4cd7-88f8-0a6ae2925601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:27.889653 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8037ee-993c-4cd7-88f8-0a6ae2925601 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:27.890368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8037ee-993c-4cd7-88f8-0a6ae2925601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:27.892399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8037ee-993c-4cd7-88f8-0a6ae2925601 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:27.935581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 907/1811] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:27 2022] POST /identity/v3/users => generated 285 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:27.937263 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8037ee-993c-4cd7-88f8-0a6ae2925601 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:27.969151 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8037ee-993c-4cd7-88f8-0a6ae2925601 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:27.987942 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa8969-90bc-4957-84ee-1f600d7ddb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:27.988599 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa8969-90bc-4957-84ee-1f600d7ddb04 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:27.988883 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa8969-90bc-4957-84ee-1f600d7ddb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:27.989160 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa8969-90bc-4957-84ee-1f600d7ddb04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:27.992303 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 905/1812] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:27.998465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44defde8-cc1f-42ca-8ec2-e31f7806d236 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:27.999751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa8969-90bc-4957-84ee-1f600d7ddb04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:28.004339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa8969-90bc-4957-84ee-1f600d7ddb04 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:28.032519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44defde8-cc1f-42ca-8ec2-e31f7806d236 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:28.037300 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 908/1813] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:28.058974 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa8969-90bc-4957-84ee-1f600d7ddb04 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:28.073597 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f905f-3df4-4885-83a4-e2970bc6e523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:28.073965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f905f-3df4-4885-83a4-e2970bc6e523 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:28.074224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f905f-3df4-4885-83a4-e2970bc6e523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:28.074527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f905f-3df4-4885-83a4-e2970bc6e523 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:28.085227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f905f-3df4-4885-83a4-e2970bc6e523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:28.091587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f905f-3df4-4885-83a4-e2970bc6e523 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:28.110880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa8969-90bc-4957-84ee-1f600d7ddb04 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:28.142492 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7cd65-7f93-45f1-82f3-001106fb7dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:28.143209 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7cd65-7f93-45f1-82f3-001106fb7dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:28.143800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7cd65-7f93-45f1-82f3-001106fb7dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:28.144077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7cd65-7f93-45f1-82f3-001106fb7dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:28.149953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7cd65-7f93-45f1-82f3-001106fb7dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:28.154943 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7cd65-7f93-45f1-82f3-001106fb7dc2 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:28.183083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 909/1814] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:28.185463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 906/1815] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:27 2022] POST /identity/v3/projects => generated 377 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:28.191016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7cd65-7f93-45f1-82f3-001106fb7dc2 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:28.193076 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f905f-3df4-4885-83a4-e2970bc6e523 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:28.213030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7cd65-7f93-45f1-82f3-001106fb7dc2 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:28.274233 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:28.347469 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f905f-3df4-4885-83a4-e2970bc6e523 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:28.360592 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb3ffa-ea3c-40d0-8acf-a12764538dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:28.361630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb3ffa-ea3c-40d0-8acf-a12764538dbf admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:28.361969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb3ffa-ea3c-40d0-8acf-a12764538dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:28.363032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb3ffa-ea3c-40d0-8acf-a12764538dbf admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:28.363754 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb3ffa-ea3c-40d0-8acf-a12764538dbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:28.367535 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb3ffa-ea3c-40d0-8acf-a12764538dbf admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:28.377652 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9608e3-ae55-477b-8ec2-28dee36bed69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:28.378505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9608e3-ae55-477b-8ec2-28dee36bed69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:28.378777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9608e3-ae55-477b-8ec2-28dee36bed69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:28.379107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9608e3-ae55-477b-8ec2-28dee36bed69 admin admin] PATH_INFO: `/v3/projects/dcd67bf7b46945e0abc7ba3544a41978/users/29ecbbb6c09d49258bfdda43436f8bfd/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:28.411828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9608e3-ae55-477b-8ec2-28dee36bed69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcd67bf7b46945e0abc7ba3544a41978, user_id=29ecbbb6c09d49258bfdda43436f8bfd, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:28.412968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 907/1816] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:28 2022] POST /identity/v3/users => generated 289 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:28.418308 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb3ffa-ea3c-40d0-8acf-a12764538dbf admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:28.419923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9608e3-ae55-477b-8ec2-28dee36bed69 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:28.446554 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 910/1817] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:28 2022] PUT /identity/v3/projects/dcd67bf7b46945e0abc7ba3544a41978/users/29ecbbb6c09d49258bfdda43436f8bfd/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:28.459392 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03ee96-c45e-4331-8793-be07a9343eb9 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:28.460581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03ee96-c45e-4331-8793-be07a9343eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:28.461511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03ee96-c45e-4331-8793-be07a9343eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:28.463451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb3ffa-ea3c-40d0-8acf-a12764538dbf admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:28.501896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-5b03ee96-c45e-4331-8793-be07a9343eb9 None None] MFA Rules not processed for user `29ecbbb6c09d49258bfdda43436f8bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:28.503428 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5b03ee96-c45e-4331-8793-be07a9343eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:28.591040 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 911/1818] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:28.596052 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03ee96-c45e-4331-8793-be07a9343eb9 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:28.596781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d8369-7568-44d4-a0fb-4e0dcb101c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:28.597701 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d8369-7568-44d4-a0fb-4e0dcb101c09 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:28.598106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d8369-7568-44d4-a0fb-4e0dcb101c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:28.598615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d8369-7568-44d4-a0fb-4e0dcb101c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:28.599871 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d8369-7568-44d4-a0fb-4e0dcb101c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:28.601657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d8369-7568-44d4-a0fb-4e0dcb101c09 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:28.611498 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03ee96-c45e-4331-8793-be07a9343eb9 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:28.618125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 908/1819] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:24:28.619326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d8369-7568-44d4-a0fb-4e0dcb101c09 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:28.620771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d0308-c4af-43b4-bfb2-ee7ff2e7565f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:28.621437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d0308-c4af-43b4-bfb2-ee7ff2e7565f admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:28.621716 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d0308-c4af-43b4-bfb2-ee7ff2e7565f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:28.622017 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d0308-c4af-43b4-bfb2-ee7ff2e7565f admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:28.622708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d0308-c4af-43b4-bfb2-ee7ff2e7565f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:28.624826 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d0308-c4af-43b4-bfb2-ee7ff2e7565f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:28.634451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d8369-7568-44d4-a0fb-4e0dcb101c09 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:28.641809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8807b2e-9166-417b-b036-cf731bfeb8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:28.642602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8807b2e-9166-417b-b036-cf731bfeb8a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:28.642983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8807b2e-9166-417b-b036-cf731bfeb8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:28.643497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8807b2e-9166-417b-b036-cf731bfeb8a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:28.644387 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8807b2e-9166-417b-b036-cf731bfeb8a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:28.647339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8807b2e-9166-417b-b036-cf731bfeb8a9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:28.665996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 909/1820] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:28.667052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8807b2e-9166-417b-b036-cf731bfeb8a9 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:28.673182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 912/1821] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:28 2022] POST /identity/v3/projects => generated 367 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:28.675818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d0308-c4af-43b4-bfb2-ee7ff2e7565f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:28.680660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8807b2e-9166-417b-b036-cf731bfeb8a9 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:28.706181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:28.780812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b673a-1bca-4804-a399-7d1d82d5f0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:28.781862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b673a-1bca-4804-a399-7d1d82d5f0a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:28.782147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b673a-1bca-4804-a399-7d1d82d5f0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:28.782424 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b673a-1bca-4804-a399-7d1d82d5f0a0 admin admin] PATH_INFO: `/v3/projects/04efe9f324e44df9b8ff0986f2599fb9/users/3490b2fcdcbc4ee9a3aaf1feed368457/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:28.810694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d0308-c4af-43b4-bfb2-ee7ff2e7565f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:28.819530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b673a-1bca-4804-a399-7d1d82d5f0a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04efe9f324e44df9b8ff0986f2599fb9, user_id=3490b2fcdcbc4ee9a3aaf1feed368457, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:28.823713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c32b2-53fa-4270-9cae-6e05ea1528fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:28.824381 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c32b2-53fa-4270-9cae-6e05ea1528fd admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:28.824663 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c32b2-53fa-4270-9cae-6e05ea1528fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:28.824939 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c32b2-53fa-4270-9cae-6e05ea1528fd admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:28.827929 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c32b2-53fa-4270-9cae-6e05ea1528fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:28.828973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b673a-1bca-4804-a399-7d1d82d5f0a0 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:28.835225 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c32b2-53fa-4270-9cae-6e05ea1528fd admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:28.858243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 910/1822] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:28 2022] PUT /identity/v3/projects/04efe9f324e44df9b8ff0986f2599fb9/users/3490b2fcdcbc4ee9a3aaf1feed368457/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:28.867555 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52640be-a878-4509-b4af-4a8045419459 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:28.868960 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52640be-a878-4509-b4af-4a8045419459 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:28.869489 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52640be-a878-4509-b4af-4a8045419459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:28.888834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 913/1823] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:28 2022] POST /identity/v3/users => generated 290 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:28.891515 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c32b2-53fa-4270-9cae-6e05ea1528fd admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:28.919600 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c32b2-53fa-4270-9cae-6e05ea1528fd admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:28.932167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c52640be-a878-4509-b4af-4a8045419459 None None] MFA Rules not processed for user `3490b2fcdcbc4ee9a3aaf1feed368457`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:28.933759 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c52640be-a878-4509-b4af-4a8045419459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:29.000875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:29.078842 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678cda0-13fb-4bd9-b889-d253dbdc7720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:29.079606 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678cda0-13fb-4bd9-b889-d253dbdc7720 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:29.079881 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678cda0-13fb-4bd9-b889-d253dbdc7720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:29.080159 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678cda0-13fb-4bd9-b889-d253dbdc7720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:29.080881 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678cda0-13fb-4bd9-b889-d253dbdc7720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:29.084306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678cda0-13fb-4bd9-b889-d253dbdc7720 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:29.085425 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 911/1824] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:29.086443 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52640be-a878-4509-b4af-4a8045419459 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:29.101446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52640be-a878-4509-b4af-4a8045419459 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:29.107506 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49017a96-8efe-4f84-975f-ab5dda93f551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:29.108321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49017a96-8efe-4f84-975f-ab5dda93f551 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:29.108610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49017a96-8efe-4f84-975f-ab5dda93f551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:29.108888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49017a96-8efe-4f84-975f-ab5dda93f551 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:29.109570 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49017a96-8efe-4f84-975f-ab5dda93f551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:29.112809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49017a96-8efe-4f84-975f-ab5dda93f551 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:29.117174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 914/1825] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:29.121681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678cda0-13fb-4bd9-b889-d253dbdc7720 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:29.146885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678cda0-13fb-4bd9-b889-d253dbdc7720 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:29.160999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a26ce0-5c1e-4c05-9577-38f726f650e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:29.162079 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a26ce0-5c1e-4c05-9577-38f726f650e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:29.162364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a26ce0-5c1e-4c05-9577-38f726f650e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:29.162639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a26ce0-5c1e-4c05-9577-38f726f650e9 admin admin] PATH_INFO: `/v3/projects/0489f12f51ca47098880337dca5eb9b6/users/dcbe7cc8dfba407188b5b14090cde8b3/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:29.173451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 912/1826] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:29 2022] POST /identity/v3/projects => generated 375 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:29.179086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49017a96-8efe-4f84-975f-ab5dda93f551 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:29.204085 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a26ce0-5c1e-4c05-9577-38f726f650e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0489f12f51ca47098880337dca5eb9b6, user_id=dcbe7cc8dfba407188b5b14090cde8b3, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:29.208878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a26ce0-5c1e-4c05-9577-38f726f650e9 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:29.242344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 915/1827] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:29 2022] PUT /identity/v3/projects/0489f12f51ca47098880337dca5eb9b6/users/dcbe7cc8dfba407188b5b14090cde8b3/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:29.243338 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a26ce0-5c1e-4c05-9577-38f726f650e9 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:29.311033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49017a96-8efe-4f84-975f-ab5dda93f551 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:29.424943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8263b7-f465-4d91-a5d4-f649223a903d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:29.425766 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8263b7-f465-4d91-a5d4-f649223a903d admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:29.426083 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8263b7-f465-4d91-a5d4-f649223a903d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:29.426367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8263b7-f465-4d91-a5d4-f649223a903d admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:29.427877 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8263b7-f465-4d91-a5d4-f649223a903d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:29.431223 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8263b7-f465-4d91-a5d4-f649223a903d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:29.435424 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a26ce0-5c1e-4c05-9577-38f726f650e9 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:29.452030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35478035-f488-48c3-963f-892fb174bb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:29.455520 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35478035-f488-48c3-963f-892fb174bb73 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:29.455907 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35478035-f488-48c3-963f-892fb174bb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:29.456224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35478035-f488-48c3-963f-892fb174bb73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:29.457137 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35478035-f488-48c3-963f-892fb174bb73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:29.459764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35478035-f488-48c3-963f-892fb174bb73 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:29.467017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 913/1828] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:29 2022] POST /identity/v3/users => generated 294 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:29.471601 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8263b7-f465-4d91-a5d4-f649223a903d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:29.490712 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 916/1829] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:29.495328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35478035-f488-48c3-963f-892fb174bb73 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:29.496827 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8263b7-f465-4d91-a5d4-f649223a903d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:29.520922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6ab67-726c-4c64-bf86-1eedf407c543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:29.521701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35478035-f488-48c3-963f-892fb174bb73 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:29.522669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6ab67-726c-4c64-bf86-1eedf407c543 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:29.522669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6ab67-726c-4c64-bf86-1eedf407c543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:29.522669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6ab67-726c-4c64-bf86-1eedf407c543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:29.523274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6ab67-726c-4c64-bf86-1eedf407c543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:29.524978 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6ab67-726c-4c64-bf86-1eedf407c543 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:29.534697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a32c9-17b5-4010-a233-c89de43d99d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:29.534945 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a32c9-17b5-4010-a233-c89de43d99d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:29.535231 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a32c9-17b5-4010-a233-c89de43d99d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:29.535456 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a32c9-17b5-4010-a233-c89de43d99d5 admin admin] PATH_INFO: `/v3/projects/0489f12f51ca47098880337dca5eb9b6/users/dcbe7cc8dfba407188b5b14090cde8b3/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:29.538157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a32c9-17b5-4010-a233-c89de43d99d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0489f12f51ca47098880337dca5eb9b6, user_id=dcbe7cc8dfba407188b5b14090cde8b3, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:29.541289 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a32c9-17b5-4010-a233-c89de43d99d5 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:29.551207 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 914/1830] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:24:29.555920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6ab67-726c-4c64-bf86-1eedf407c543 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:29.559789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:29.567825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 917/1831] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:29 2022] PUT /identity/v3/projects/0489f12f51ca47098880337dca5eb9b6/users/dcbe7cc8dfba407188b5b14090cde8b3/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:29.569223 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6ab67-726c-4c64-bf86-1eedf407c543 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:29.581587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16405bd-d903-47f8-b1cb-46600a26f0b8 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:29.582119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16405bd-d903-47f8-b1cb-46600a26f0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:29.582499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16405bd-d903-47f8-b1cb-46600a26f0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:29.615463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:29.624435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-d16405bd-d903-47f8-b1cb-46600a26f0b8 None None] MFA Rules not processed for user `dcbe7cc8dfba407188b5b14090cde8b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:29.626042 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d16405bd-d903-47f8-b1cb-46600a26f0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:29.688039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fd253-4a67-4f64-af49-84aaa9f77ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:29.688816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fd253-4a67-4f64-af49-84aaa9f77ce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:29.689425 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fd253-4a67-4f64-af49-84aaa9f77ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:29.689715 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fd253-4a67-4f64-af49-84aaa9f77ce9 admin admin] PATH_INFO: `/v3/projects/aa50cc8d209c4b9d8fea46a1e23d7605/users/a6c7bf11ee03404598e54b05c2332090/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:29.713306 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596fd253-4a67-4f64-af49-84aaa9f77ce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa50cc8d209c4b9d8fea46a1e23d7605, user_id=a6c7bf11ee03404598e54b05c2332090, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:29.718243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596fd253-4a67-4f64-af49-84aaa9f77ce9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:29.742844 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 918/1832] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:29 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:29.743976 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 915/1833] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:29 2022] PUT /identity/v3/projects/aa50cc8d209c4b9d8fea46a1e23d7605/users/a6c7bf11ee03404598e54b05c2332090/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:24:29.759556 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fd253-4a67-4f64-af49-84aaa9f77ce9 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:29.760741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16405bd-d903-47f8-b1cb-46600a26f0b8 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:29.761548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-596fd253-4a67-4f64-af49-84aaa9f77ce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:29.784946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16405bd-d903-47f8-b1cb-46600a26f0b8 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:29.883063 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25c2ed-7731-4441-a36f-503a1f59e04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:29.883751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25c2ed-7731-4441-a36f-503a1f59e04e admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:29.885201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25c2ed-7731-4441-a36f-503a1f59e04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:29.885563 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25c2ed-7731-4441-a36f-503a1f59e04e admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:29.886627 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25c2ed-7731-4441-a36f-503a1f59e04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:29.888235 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25c2ed-7731-4441-a36f-503a1f59e04e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:29.892395 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596fd253-4a67-4f64-af49-84aaa9f77ce9 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:29.901902 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d12ab-2e2f-4d39-8db7-bec35c312042 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:29.902610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d12ab-2e2f-4d39-8db7-bec35c312042 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:29.902891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d12ab-2e2f-4d39-8db7-bec35c312042 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:29.903157 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d12ab-2e2f-4d39-8db7-bec35c312042 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:29.914163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 919/1834] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:29.914163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25c2ed-7731-4441-a36f-503a1f59e04e admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:29.932599 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25c2ed-7731-4441-a36f-503a1f59e04e admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:29.953016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec506d-67cf-49cc-8dab-fa3c50ab3c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:29.953834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec506d-67cf-49cc-8dab-fa3c50ab3c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:29.954314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec506d-67cf-49cc-8dab-fa3c50ab3c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:29.954601 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec506d-67cf-49cc-8dab-fa3c50ab3c8e admin admin] PATH_INFO: `/v3/projects/aa50cc8d209c4b9d8fea46a1e23d7605/users/a6c7bf11ee03404598e54b05c2332090/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:29.967800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec506d-67cf-49cc-8dab-fa3c50ab3c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa50cc8d209c4b9d8fea46a1e23d7605, user_id=a6c7bf11ee03404598e54b05c2332090, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:29.971680 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec506d-67cf-49cc-8dab-fa3c50ab3c8e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:30.008264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 920/1835] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:29 2022] PUT /identity/v3/projects/aa50cc8d209c4b9d8fea46a1e23d7605/users/a6c7bf11ee03404598e54b05c2332090/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:24:30.019282 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbba73-742a-4c18-b252-f7cd68060815 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:30.019568 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbba73-742a-4c18-b252-f7cd68060815 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:30.019903 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbba73-742a-4c18-b252-f7cd68060815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:30.069468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-42cbba73-742a-4c18-b252-f7cd68060815 None None] MFA Rules not processed for user `a6c7bf11ee03404598e54b05c2332090`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:30.071730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-42cbba73-742a-4c18-b252-f7cd68060815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:30.080988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d12ab-2e2f-4d39-8db7-bec35c312042 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:30.083274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d12ab-2e2f-4d39-8db7-bec35c312042 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:30.112564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 916/1836] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:29 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:24:30.174063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d12ab-2e2f-4d39-8db7-bec35c312042 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:30.176509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3d6d12ab-2e2f-4d39-8db7-bec35c312042 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:30.183232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:30.195335 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d12ab-2e2f-4d39-8db7-bec35c312042 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:30.209405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 921/1837] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:30.217869 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cbba73-742a-4c18-b252-f7cd68060815 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:30.280412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c746a2f-4a32-4c0c-90aa-eb849fa8bb4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:30.282494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c746a2f-4a32-4c0c-90aa-eb849fa8bb4d service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:30.283467 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c746a2f-4a32-4c0c-90aa-eb849fa8bb4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:30.284249 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c746a2f-4a32-4c0c-90aa-eb849fa8bb4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:30.292044 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cbba73-742a-4c18-b252-f7cd68060815 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:30.296982 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c06a0c-dec6-4e52-8e82-fabcd486b49e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:30.298223 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c06a0c-dec6-4e52-8e82-fabcd486b49e service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:30.298618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c06a0c-dec6-4e52-8e82-fabcd486b49e service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:30.299035 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c06a0c-dec6-4e52-8e82-fabcd486b49e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:30.303896 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:30.394082 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c06a0c-dec6-4e52-8e82-fabcd486b49e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:30.395761 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c06a0c-dec6-4e52-8e82-fabcd486b49e service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:30.409383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 922/1838] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:30.446139 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c746a2f-4a32-4c0c-90aa-eb849fa8bb4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:30.452617 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c746a2f-4a32-4c0c-90aa-eb849fa8bb4d service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:30.470816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 917/1839] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:30 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:31.808088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5396b5b-f512-42bb-9e9f-23e27520224c None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:31.808653 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5396b5b-f512-42bb-9e9f-23e27520224c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:31.808653 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5396b5b-f512-42bb-9e9f-23e27520224c None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:31.810202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 923/1840] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 15:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:24:31.827607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee62e99-be64-4c39-9483-fe7407593846 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:31.828164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee62e99-be64-4c39-9483-fe7407593846 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:31.828164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee62e99-be64-4c39-9483-fe7407593846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:31.854871 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3ee62e99-be64-4c39-9483-fe7407593846 None None] MFA Rules not processed for user `2de0ff99a2b74979a28a8b10e9ebdfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:31.856168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3ee62e99-be64-4c39-9483-fe7407593846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:31.968501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 918/1841] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 15:24:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:32.098823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5396b5b-f512-42bb-9e9f-23e27520224c None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:32.102751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-b5396b5b-f512-42bb-9e9f-23e27520224c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:32.141633 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5396b5b-f512-42bb-9e9f-23e27520224c None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:32.151338 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49511f-da51-4f38-a2c5-ac6d04951952 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:32.152334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49511f-da51-4f38-a2c5-ac6d04951952 service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:32.152805 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49511f-da51-4f38-a2c5-ac6d04951952 service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:32.154270 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49511f-da51-4f38-a2c5-ac6d04951952 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:32.172956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49511f-da51-4f38-a2c5-ac6d04951952 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:32.179567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49511f-da51-4f38-a2c5-ac6d04951952 service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:32.231865 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 924/1842] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:32.606934 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee62e99-be64-4c39-9483-fe7407593846 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:32.619997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee62e99-be64-4c39-9483-fe7407593846 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:32.625670 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d92f82-245b-4b73-8bde-08e0ed3a6b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:32.626469 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d92f82-245b-4b73-8bde-08e0ed3a6b30 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:32.626469 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d92f82-245b-4b73-8bde-08e0ed3a6b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:32.627540 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d92f82-245b-4b73-8bde-08e0ed3a6b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:32.703376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d92f82-245b-4b73-8bde-08e0ed3a6b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:32.705444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d92f82-245b-4b73-8bde-08e0ed3a6b30 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:32.718242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 919/1843] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:33.472587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49511f-da51-4f38-a2c5-ac6d04951952 service nova] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:33.483179 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49511f-da51-4f38-a2c5-ac6d04951952 service nova] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:33.489235 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27193136-60a2-441c-b64f-8c66f14d32bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:33.489958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27193136-60a2-441c-b64f-8c66f14d32bf service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:33.490242 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27193136-60a2-441c-b64f-8c66f14d32bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:33.490509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27193136-60a2-441c-b64f-8c66f14d32bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:33.501666 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27193136-60a2-441c-b64f-8c66f14d32bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:33.503426 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27193136-60a2-441c-b64f-8c66f14d32bf service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:33.514976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 925/1844] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:34.801189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d92f82-245b-4b73-8bde-08e0ed3a6b30 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:34.856538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:34.879787 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d92f82-245b-4b73-8bde-08e0ed3a6b30 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:34.885412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692dc44-e39f-4920-adea-3fb956149bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:34.886107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692dc44-e39f-4920-adea-3fb956149bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:34.886387 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692dc44-e39f-4920-adea-3fb956149bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:34.886658 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692dc44-e39f-4920-adea-3fb956149bb1 admin admin] PATH_INFO: `/v3/users/29ecbbb6c09d49258bfdda43436f8bfd` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:34.887816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692dc44-e39f-4920-adea-3fb956149bb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29ecbbb6c09d49258bfdda43436f8bfd)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:34.889756 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0692dc44-e39f-4920-adea-3fb956149bb1 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:34.952554 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0692dc44-e39f-4920-adea-3fb956149bb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29ecbbb6c09d49258bfdda43436f8bfd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:34.953326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0692dc44-e39f-4920-adea-3fb956149bb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29ecbbb6c09d49258bfdda43436f8bfd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:34.971304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0692dc44-e39f-4920-adea-3fb956149bb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29ecbbb6c09d49258bfdda43436f8bfd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:34.984302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0692dc44-e39f-4920-adea-3fb956149bb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29ecbbb6c09d49258bfdda43436f8bfd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:34.994522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0692dc44-e39f-4920-adea-3fb956149bb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29ecbbb6c09d49258bfdda43436f8bfd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:35.001274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 920/1845] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:34 2022] DELETE /identity/v3/users/29ecbbb6c09d49258bfdda43436f8bfd => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:35.006303 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd7bbb-3a10-4e38-ba8d-747c3b1defb9 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:35.006935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd7bbb-3a10-4e38-ba8d-747c3b1defb9 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:35.006935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd7bbb-3a10-4e38-ba8d-747c3b1defb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:35.033656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-5fdd7bbb-3a10-4e38-ba8d-747c3b1defb9 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:35.034971 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5fdd7bbb-3a10-4e38-ba8d-747c3b1defb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:35.118121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 926/1846] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:35.128197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692dc44-e39f-4920-adea-3fb956149bb1 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:35.130047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0692dc44-e39f-4920-adea-3fb956149bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:35.201072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692dc44-e39f-4920-adea-3fb956149bb1 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:35.204986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd16b64-2067-4ea1-bf01-b279869ec6b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:35.205445 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd16b64-2067-4ea1-bf01-b279869ec6b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:35.205726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd16b64-2067-4ea1-bf01-b279869ec6b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:35.206043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd16b64-2067-4ea1-bf01-b279869ec6b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:35.216633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd16b64-2067-4ea1-bf01-b279869ec6b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:35.218151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd16b64-2067-4ea1-bf01-b279869ec6b4 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:35.229107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 921/1847] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:35.684088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd7bbb-3a10-4e38-ba8d-747c3b1defb9 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:35.686448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5fdd7bbb-3a10-4e38-ba8d-747c3b1defb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:35.711832 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd7bbb-3a10-4e38-ba8d-747c3b1defb9 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:35.718108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bbf77-3a74-47f7-b64f-483c1c560694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:35.718901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bbf77-3a74-47f7-b64f-483c1c560694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:35.719274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bbf77-3a74-47f7-b64f-483c1c560694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:35.719637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bbf77-3a74-47f7-b64f-483c1c560694 admin admin] PATH_INFO: `/v3/projects/dcd67bf7b46945e0abc7ba3544a41978` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:35.720979 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bbf77-3a74-47f7-b64f-483c1c560694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcd67bf7b46945e0abc7ba3544a41978)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:35.722951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bbf77-3a74-47f7-b64f-483c1c560694 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:35.741859 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:35.781965 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-bb1bbf77-3a74-47f7-b64f-483c1c560694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcd67bf7b46945e0abc7ba3544a41978', 'request_id': 'req-bb1bbf77-3a74-47f7-b64f-483c1c560694'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:35.792671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-bb1bbf77-3a74-47f7-b64f-483c1c560694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcd67bf7b46945e0abc7ba3544a41978', 'request_id': 'req-bb1bbf77-3a74-47f7-b64f-483c1c560694'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:35.802457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-bb1bbf77-3a74-47f7-b64f-483c1c560694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcd67bf7b46945e0abc7ba3544a41978', 'request_id': 'req-bb1bbf77-3a74-47f7-b64f-483c1c560694'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:35.815457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-bb1bbf77-3a74-47f7-b64f-483c1c560694 admin admin] The token cache is being invalidate because project dcd67bf7b46945e0abc7ba3544a41978 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:35.815741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-bb1bbf77-3a74-47f7-b64f-483c1c560694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:35.816400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-bb1bbf77-3a74-47f7-b64f-483c1c560694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:35.818154 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 927/1848] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:35 2022] DELETE /identity/v3/projects/dcd67bf7b46945e0abc7ba3544a41978 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:35.823227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd16b64-2067-4ea1-bf01-b279869ec6b4 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:35.841568 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd16b64-2067-4ea1-bf01-b279869ec6b4 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:35.921993 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caae2d4-354e-4ee1-8627-f94354ffab36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:35.922690 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caae2d4-354e-4ee1-8627-f94354ffab36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:35.922976 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caae2d4-354e-4ee1-8627-f94354ffab36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:35.923254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caae2d4-354e-4ee1-8627-f94354ffab36 admin admin] PATH_INFO: `/v3/users/dcbe7cc8dfba407188b5b14090cde8b3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:35.924502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caae2d4-354e-4ee1-8627-f94354ffab36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcbe7cc8dfba407188b5b14090cde8b3)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:35.926526 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caae2d4-354e-4ee1-8627-f94354ffab36 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:35.938094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:35.994756 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8caae2d4-354e-4ee1-8627-f94354ffab36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcbe7cc8dfba407188b5b14090cde8b3'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:35.994756 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8caae2d4-354e-4ee1-8627-f94354ffab36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcbe7cc8dfba407188b5b14090cde8b3'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:36.013501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8caae2d4-354e-4ee1-8627-f94354ffab36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcbe7cc8dfba407188b5b14090cde8b3'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:36.025225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8caae2d4-354e-4ee1-8627-f94354ffab36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcbe7cc8dfba407188b5b14090cde8b3'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:36.035673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8caae2d4-354e-4ee1-8627-f94354ffab36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcbe7cc8dfba407188b5b14090cde8b3'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:36.042210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 922/1849] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:35 2022] DELETE /identity/v3/users/dcbe7cc8dfba407188b5b14090cde8b3 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:36.335025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bbf77-3a74-47f7-b64f-483c1c560694 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:36.336643 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-bb1bbf77-3a74-47f7-b64f-483c1c560694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:36.353683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bbf77-3a74-47f7-b64f-483c1c560694 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:36.426904 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66faab-15c3-46f2-bf9d-34461c1bf26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:36.427695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66faab-15c3-46f2-bf9d-34461c1bf26a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:36.427975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66faab-15c3-46f2-bf9d-34461c1bf26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:36.428304 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66faab-15c3-46f2-bf9d-34461c1bf26a admin admin] PATH_INFO: `/v3/projects/0489f12f51ca47098880337dca5eb9b6` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:36.429523 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66faab-15c3-46f2-bf9d-34461c1bf26a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0489f12f51ca47098880337dca5eb9b6)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:36.431582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66faab-15c3-46f2-bf9d-34461c1bf26a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:36.480150 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ee66faab-15c3-46f2-bf9d-34461c1bf26a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0489f12f51ca47098880337dca5eb9b6', 'request_id': 'req-ee66faab-15c3-46f2-bf9d-34461c1bf26a'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:36.489636 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ee66faab-15c3-46f2-bf9d-34461c1bf26a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0489f12f51ca47098880337dca5eb9b6', 'request_id': 'req-ee66faab-15c3-46f2-bf9d-34461c1bf26a'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:36.497168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ee66faab-15c3-46f2-bf9d-34461c1bf26a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0489f12f51ca47098880337dca5eb9b6', 'request_id': 'req-ee66faab-15c3-46f2-bf9d-34461c1bf26a'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:36.510036 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ee66faab-15c3-46f2-bf9d-34461c1bf26a admin admin] The token cache is being invalidate because project 0489f12f51ca47098880337dca5eb9b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:36.510225 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ee66faab-15c3-46f2-bf9d-34461c1bf26a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:36.510909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ee66faab-15c3-46f2-bf9d-34461c1bf26a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:36.512449 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 928/1850] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:36 2022] DELETE /identity/v3/projects/0489f12f51ca47098880337dca5eb9b6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:36.517450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1c345-7065-48a6-8e3e-fcf23ac386a0 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:36.517806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1c345-7065-48a6-8e3e-fcf23ac386a0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:36.518112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1c345-7065-48a6-8e3e-fcf23ac386a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:36.540111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-bfe1c345-7065-48a6-8e3e-fcf23ac386a0 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:36.541457 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bfe1c345-7065-48a6-8e3e-fcf23ac386a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:36.611853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 923/1851] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:36.622762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb08cf-5cde-4182-8510-ece7f82907db None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:36.623164 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb08cf-5cde-4182-8510-ece7f82907db None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:36.623594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb08cf-5cde-4182-8510-ece7f82907db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:36.653136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-c0eb08cf-5cde-4182-8510-ece7f82907db None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:36.654408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c0eb08cf-5cde-4182-8510-ece7f82907db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:36.665336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 929/1852] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:36.670118 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1c345-7065-48a6-8e3e-fcf23ac386a0 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:36.680552 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1c345-7065-48a6-8e3e-fcf23ac386a0 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:36.685835 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddbb83-a63c-4548-8801-263c0f410a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:36.686719 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddbb83-a63c-4548-8801-263c0f410a24 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:36.686719 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddbb83-a63c-4548-8801-263c0f410a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:36.687013 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddbb83-a63c-4548-8801-263c0f410a24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:36.687724 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ddbb83-a63c-4548-8801-263c0f410a24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:24:36.688023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ddbb83-a63c-4548-8801-263c0f410a24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:36.689954 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ddbb83-a63c-4548-8801-263c0f410a24 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:36.703189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 924/1853] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:24:36.707947 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb08cf-5cde-4182-8510-ece7f82907db None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:36.718151 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb08cf-5cde-4182-8510-ece7f82907db None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:36.722974 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eeccf8-46a6-4a77-b83d-30bd00234599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:36.723618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeccf8-46a6-4a77-b83d-30bd00234599 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:36.723896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeccf8-46a6-4a77-b83d-30bd00234599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:36.724171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeccf8-46a6-4a77-b83d-30bd00234599 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:36.724856 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eeccf8-46a6-4a77-b83d-30bd00234599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:36.726862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eeccf8-46a6-4a77-b83d-30bd00234599 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:36.774776 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 930/1854] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:36 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:36.776211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddbb83-a63c-4548-8801-263c0f410a24 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:36.788261 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddbb83-a63c-4548-8801-263c0f410a24 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:36.877890 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d6fbd-0c7d-4631-9990-08f590edc12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:36.878566 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d6fbd-0c7d-4631-9990-08f590edc12a admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:36.878804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d6fbd-0c7d-4631-9990-08f590edc12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:36.879125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d6fbd-0c7d-4631-9990-08f590edc12a admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:36.879734 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d6fbd-0c7d-4631-9990-08f590edc12a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:36.881725 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d6fbd-0c7d-4631-9990-08f590edc12a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:36.909150 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 925/1855] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:36 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:36.910481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eeccf8-46a6-4a77-b83d-30bd00234599 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:36.923513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eeccf8-46a6-4a77-b83d-30bd00234599 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:36.929873 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f192b82-c478-447d-9bc4-2a139d8b0641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:36.930603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f192b82-c478-447d-9bc4-2a139d8b0641 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:36.930881 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f192b82-c478-447d-9bc4-2a139d8b0641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:36.931153 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f192b82-c478-447d-9bc4-2a139d8b0641 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:36.931863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f192b82-c478-447d-9bc4-2a139d8b0641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:36.933579 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f192b82-c478-447d-9bc4-2a139d8b0641 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:36.935132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:36.945879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 931/1856] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:36.951222 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d6fbd-0c7d-4631-9990-08f590edc12a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:36.960132 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:36.965174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d6fbd-0c7d-4631-9990-08f590edc12a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:36.971601 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1c773-4247-4ec7-91ce-6e6f58fa0387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:36.972328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1c773-4247-4ec7-91ce-6e6f58fa0387 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:36.972845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1c773-4247-4ec7-91ce-6e6f58fa0387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:36.973426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1c773-4247-4ec7-91ce-6e6f58fa0387 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:36.974286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1c773-4247-4ec7-91ce-6e6f58fa0387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:36.976050 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d1c773-4247-4ec7-91ce-6e6f58fa0387 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:36.990196 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 926/1857] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:36.994876 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f192b82-c478-447d-9bc4-2a139d8b0641 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:37.005909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f192b82-c478-447d-9bc4-2a139d8b0641 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:37.012617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5858fe-df8d-419b-9653-eb7a5d4863a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:37.013249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5858fe-df8d-419b-9653-eb7a5d4863a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:37.013530 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5858fe-df8d-419b-9653-eb7a5d4863a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:37.013840 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5858fe-df8d-419b-9653-eb7a5d4863a9 admin admin] PATH_INFO: `/v3/projects/6ed60f536b0f4e93a27232ce2f841c69/users/4403f58140e3439b8375aa4e5e93d9ff/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:37.031415 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5858fe-df8d-419b-9653-eb7a5d4863a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed60f536b0f4e93a27232ce2f841c69, user_id=4403f58140e3439b8375aa4e5e93d9ff, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:37.034299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5858fe-df8d-419b-9653-eb7a5d4863a9 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:37.055248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 932/1858] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:36 2022] PUT /identity/v3/projects/6ed60f536b0f4e93a27232ce2f841c69/users/4403f58140e3439b8375aa4e5e93d9ff/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:37.067222 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b517d4-5f9f-418a-999f-99b5260fd7a4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:37.068841 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b517d4-5f9f-418a-999f-99b5260fd7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:37.069605 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b517d4-5f9f-418a-999f-99b5260fd7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:37.136467 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-96b517d4-5f9f-418a-999f-99b5260fd7a4 None None] MFA Rules not processed for user `4403f58140e3439b8375aa4e5e93d9ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:37.137361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-96b517d4-5f9f-418a-999f-99b5260fd7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:37.300479 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5858fe-df8d-419b-9653-eb7a5d4863a9 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:37.302422 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 927/1859] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:37.408440 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5858fe-df8d-419b-9653-eb7a5d4863a9 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:37.412964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041c5b7-33c9-44e4-99ae-6dac0ca200e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:37.414043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041c5b7-33c9-44e4-99ae-6dac0ca200e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:37.414241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041c5b7-33c9-44e4-99ae-6dac0ca200e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:37.414579 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041c5b7-33c9-44e4-99ae-6dac0ca200e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:37.415314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041c5b7-33c9-44e4-99ae-6dac0ca200e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:37.417558 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041c5b7-33c9-44e4-99ae-6dac0ca200e4 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:37.446756 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 933/1860] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:37 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:24:37.450776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b517d4-5f9f-418a-999f-99b5260fd7a4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:37.464111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b517d4-5f9f-418a-999f-99b5260fd7a4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:37.527607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b8dc9-8ea4-42f2-8a2f-bb397a8adc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:37.528161 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b8dc9-8ea4-42f2-8a2f-bb397a8adc6b admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:37.528453 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b8dc9-8ea4-42f2-8a2f-bb397a8adc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:37.528825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b8dc9-8ea4-42f2-8a2f-bb397a8adc6b admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:37.529371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b8dc9-8ea4-42f2-8a2f-bb397a8adc6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:37.531332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b8dc9-8ea4-42f2-8a2f-bb397a8adc6b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:37.557401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 928/1861] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:37 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:37.558647 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041c5b7-33c9-44e4-99ae-6dac0ca200e4 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:37.572397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041c5b7-33c9-44e4-99ae-6dac0ca200e4 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:37.577700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673a952-5e9e-4167-a549-d44f0b04fa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:37.578407 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673a952-5e9e-4167-a549-d44f0b04fa41 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:37.578685 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673a952-5e9e-4167-a549-d44f0b04fa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:37.578955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673a952-5e9e-4167-a549-d44f0b04fa41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:37.579697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673a952-5e9e-4167-a549-d44f0b04fa41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:37.581322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673a952-5e9e-4167-a549-d44f0b04fa41 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:37.596861 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 934/1862] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:37.598089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b8dc9-8ea4-42f2-8a2f-bb397a8adc6b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:37.608616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b8dc9-8ea4-42f2-8a2f-bb397a8adc6b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:37.614583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d54173-618f-49f6-b73b-d301ef7ec674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:37.615575 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d54173-618f-49f6-b73b-d301ef7ec674 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:37.616106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d54173-618f-49f6-b73b-d301ef7ec674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:37.616668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d54173-618f-49f6-b73b-d301ef7ec674 admin admin] PATH_INFO: `/v3/projects/19de4f1211df4bcd91c379aeaddffff2/users/e823d38797d2418ea0d1964c189027ba/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:37.634931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d54173-618f-49f6-b73b-d301ef7ec674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19de4f1211df4bcd91c379aeaddffff2, user_id=e823d38797d2418ea0d1964c189027ba, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:37.637769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d54173-618f-49f6-b73b-d301ef7ec674 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:37.653637 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 929/1863] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:37 2022] PUT /identity/v3/projects/19de4f1211df4bcd91c379aeaddffff2/users/e823d38797d2418ea0d1964c189027ba/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:37.657468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673a952-5e9e-4167-a549-d44f0b04fa41 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:37.668873 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673a952-5e9e-4167-a549-d44f0b04fa41 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:37.728396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cd69f-fc30-4eb4-869f-8e30a0d115af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:37.729031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cd69f-fc30-4eb4-869f-8e30a0d115af admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:37.729329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cd69f-fc30-4eb4-869f-8e30a0d115af admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:37.729598 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cd69f-fc30-4eb4-869f-8e30a0d115af admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:37.730320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cd69f-fc30-4eb4-869f-8e30a0d115af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:37.731942 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cd69f-fc30-4eb4-869f-8e30a0d115af admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:37.742172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 935/1864] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:24:37.746837 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d54173-618f-49f6-b73b-d301ef7ec674 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:37.758894 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d54173-618f-49f6-b73b-d301ef7ec674 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:37.765856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e3839-3e01-443c-8e88-f8b5948bdcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:37.766547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e3839-3e01-443c-8e88-f8b5948bdcba admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:37.766828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e3839-3e01-443c-8e88-f8b5948bdcba admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:37.767100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e3839-3e01-443c-8e88-f8b5948bdcba admin admin] PATH_INFO: `/v3/projects/19de4f1211df4bcd91c379aeaddffff2/users/e823d38797d2418ea0d1964c189027ba/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:37.769457 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e3839-3e01-443c-8e88-f8b5948bdcba admin admin] RBAC: Authorizing `identity:create_grant(project_id=19de4f1211df4bcd91c379aeaddffff2, user_id=e823d38797d2418ea0d1964c189027ba, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:37.772497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e3839-3e01-443c-8e88-f8b5948bdcba admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:37.783437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 930/1865] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:37 2022] PUT /identity/v3/projects/19de4f1211df4bcd91c379aeaddffff2/users/e823d38797d2418ea0d1964c189027ba/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:37.795800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87577345-bb47-4aa2-9b60-7ebb4d3a2248 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:37.796148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87577345-bb47-4aa2-9b60-7ebb4d3a2248 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:37.796516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87577345-bb47-4aa2-9b60-7ebb4d3a2248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:37.833115 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-87577345-bb47-4aa2-9b60-7ebb4d3a2248 None None] MFA Rules not processed for user `e823d38797d2418ea0d1964c189027ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:37.834440 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-87577345-bb47-4aa2-9b60-7ebb4d3a2248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:37.869773 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:37.918347 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 936/1866] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:37.930176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e3839-3e01-443c-8e88-f8b5948bdcba admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:37.932055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-973e3839-3e01-443c-8e88-f8b5948bdcba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:37.978403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:38.005181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e3839-3e01-443c-8e88-f8b5948bdcba admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:38.009737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ff98c-a972-499a-8c23-ee9fc36b8c0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:38.010426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ff98c-a972-499a-8c23-ee9fc36b8c0b service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:38.010702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ff98c-a972-499a-8c23-ee9fc36b8c0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:38.010973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ff98c-a972-499a-8c23-ee9fc36b8c0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:38.020618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ff98c-a972-499a-8c23-ee9fc36b8c0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:38.022062 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ff98c-a972-499a-8c23-ee9fc36b8c0b service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:38.032668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 931/1867] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:38.062062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87577345-bb47-4aa2-9b60-7ebb4d3a2248 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:38.126410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87577345-bb47-4aa2-9b60-7ebb4d3a2248 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:38.131620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f307f-279d-4309-9c97-327f41658345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:38.131955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f307f-279d-4309-9c97-327f41658345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:38.132241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f307f-279d-4309-9c97-327f41658345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:38.132516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f307f-279d-4309-9c97-327f41658345 admin admin] PATH_INFO: `/v3/users/4403f58140e3439b8375aa4e5e93d9ff` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:38.133749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f307f-279d-4309-9c97-327f41658345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4403f58140e3439b8375aa4e5e93d9ff)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:38.135716 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f307f-279d-4309-9c97-327f41658345 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:38.197637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-2f0f307f-279d-4309-9c97-327f41658345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4403f58140e3439b8375aa4e5e93d9ff'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:38.206618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-2f0f307f-279d-4309-9c97-327f41658345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4403f58140e3439b8375aa4e5e93d9ff'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:38.207227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-2f0f307f-279d-4309-9c97-327f41658345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4403f58140e3439b8375aa4e5e93d9ff'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:38.215851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-2f0f307f-279d-4309-9c97-327f41658345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4403f58140e3439b8375aa4e5e93d9ff'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:38.229401 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-2f0f307f-279d-4309-9c97-327f41658345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4403f58140e3439b8375aa4e5e93d9ff'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:38.237304 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 937/1868] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:38 2022] DELETE /identity/v3/users/4403f58140e3439b8375aa4e5e93d9ff => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:38.241527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a4a94-7f35-453a-af88-bafe04a5b2b6 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:38.241856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a4a94-7f35-453a-af88-bafe04a5b2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:38.242145 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a4a94-7f35-453a-af88-bafe04a5b2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:38.262494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5b7a4a94-7f35-453a-af88-bafe04a5b2b6 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:38.263723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5b7a4a94-7f35-453a-af88-bafe04a5b2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:38.349177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 932/1869] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:24:38.356468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f307f-279d-4309-9c97-327f41658345 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:38.359073 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2f0f307f-279d-4309-9c97-327f41658345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:38.431499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:38.440670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f307f-279d-4309-9c97-327f41658345 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:38.448190 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237b412-bee8-4401-923b-347053d5667c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:38.448929 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237b412-bee8-4401-923b-347053d5667c service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:38.449271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237b412-bee8-4401-923b-347053d5667c service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:38.450382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237b412-bee8-4401-923b-347053d5667c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:38.468527 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237b412-bee8-4401-923b-347053d5667c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:38.470160 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237b412-bee8-4401-923b-347053d5667c service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:38.482238 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 938/1870] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:38.960322 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a4a94-7f35-453a-af88-bafe04a5b2b6 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:38.962322 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5b7a4a94-7f35-453a-af88-bafe04a5b2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:38.975916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a4a94-7f35-453a-af88-bafe04a5b2b6 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:38.981980 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184ff44-3828-4ce2-85aa-14795df4f655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:38.982653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ff44-3828-4ce2-85aa-14795df4f655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:38.982904 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ff44-3828-4ce2-85aa-14795df4f655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:38.983181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184ff44-3828-4ce2-85aa-14795df4f655 admin admin] PATH_INFO: `/v3/projects/6ed60f536b0f4e93a27232ce2f841c69` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:38.984373 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184ff44-3828-4ce2-85aa-14795df4f655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ed60f536b0f4e93a27232ce2f841c69)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:38.986453 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184ff44-3828-4ce2-85aa-14795df4f655 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:39.037942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3184ff44-3828-4ce2-85aa-14795df4f655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ed60f536b0f4e93a27232ce2f841c69', 'request_id': 'req-3184ff44-3828-4ce2-85aa-14795df4f655'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:39.044517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3184ff44-3828-4ce2-85aa-14795df4f655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ed60f536b0f4e93a27232ce2f841c69', 'request_id': 'req-3184ff44-3828-4ce2-85aa-14795df4f655'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:39.054041 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3184ff44-3828-4ce2-85aa-14795df4f655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ed60f536b0f4e93a27232ce2f841c69', 'request_id': 'req-3184ff44-3828-4ce2-85aa-14795df4f655'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:39.068764 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3184ff44-3828-4ce2-85aa-14795df4f655 admin admin] The token cache is being invalidate because project 6ed60f536b0f4e93a27232ce2f841c69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:39.069084 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3184ff44-3828-4ce2-85aa-14795df4f655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:39.070212 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3184ff44-3828-4ce2-85aa-14795df4f655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:39.072507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 933/1871] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:38 2022] DELETE /identity/v3/projects/6ed60f536b0f4e93a27232ce2f841c69 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:39.076775 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237b412-bee8-4401-923b-347053d5667c service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:39.087041 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237b412-bee8-4401-923b-347053d5667c service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:39.154158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f318dc-4156-4b94-9cfc-40e2063ca625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:39.154818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f318dc-4156-4b94-9cfc-40e2063ca625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:39.155935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f318dc-4156-4b94-9cfc-40e2063ca625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:39.155935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f318dc-4156-4b94-9cfc-40e2063ca625 admin admin] PATH_INFO: `/v3/users/e823d38797d2418ea0d1964c189027ba` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:39.158761 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f318dc-4156-4b94-9cfc-40e2063ca625 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e823d38797d2418ea0d1964c189027ba)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:39.160898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f318dc-4156-4b94-9cfc-40e2063ca625 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:39.226792 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-f5f318dc-4156-4b94-9cfc-40e2063ca625 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e823d38797d2418ea0d1964c189027ba'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:39.238546 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-f5f318dc-4156-4b94-9cfc-40e2063ca625 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e823d38797d2418ea0d1964c189027ba'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:39.239188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-f5f318dc-4156-4b94-9cfc-40e2063ca625 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e823d38797d2418ea0d1964c189027ba'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:39.247737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-f5f318dc-4156-4b94-9cfc-40e2063ca625 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e823d38797d2418ea0d1964c189027ba'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:39.259754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-f5f318dc-4156-4b94-9cfc-40e2063ca625 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e823d38797d2418ea0d1964c189027ba'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:39.266614 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 939/1872] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:39 2022] DELETE /identity/v3/users/e823d38797d2418ea0d1964c189027ba => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:39.602957 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184ff44-3828-4ce2-85aa-14795df4f655 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:39.604553 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3184ff44-3828-4ce2-85aa-14795df4f655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:39.624357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184ff44-3828-4ce2-85aa-14795df4f655 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:39.703326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01a606-38df-48a3-a8ab-3309f689688b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:39.703997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01a606-38df-48a3-a8ab-3309f689688b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:39.704272 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01a606-38df-48a3-a8ab-3309f689688b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:39.704550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01a606-38df-48a3-a8ab-3309f689688b admin admin] PATH_INFO: `/v3/projects/19de4f1211df4bcd91c379aeaddffff2` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:39.705955 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01a606-38df-48a3-a8ab-3309f689688b admin admin] RBAC: Authorizing `identity:delete_project(project_id=19de4f1211df4bcd91c379aeaddffff2)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:39.707920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01a606-38df-48a3-a8ab-3309f689688b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:39.710610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:39.781499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cd01a606-38df-48a3-a8ab-3309f689688b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19de4f1211df4bcd91c379aeaddffff2', 'request_id': 'req-cd01a606-38df-48a3-a8ab-3309f689688b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:39.789676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cd01a606-38df-48a3-a8ab-3309f689688b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19de4f1211df4bcd91c379aeaddffff2', 'request_id': 'req-cd01a606-38df-48a3-a8ab-3309f689688b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:39.803700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cd01a606-38df-48a3-a8ab-3309f689688b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19de4f1211df4bcd91c379aeaddffff2', 'request_id': 'req-cd01a606-38df-48a3-a8ab-3309f689688b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:39.819404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cd01a606-38df-48a3-a8ab-3309f689688b admin admin] The token cache is being invalidate because project 19de4f1211df4bcd91c379aeaddffff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:39.819863 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cd01a606-38df-48a3-a8ab-3309f689688b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:39.820672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cd01a606-38df-48a3-a8ab-3309f689688b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:39.822506 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 934/1873] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:39 2022] DELETE /identity/v3/projects/19de4f1211df4bcd91c379aeaddffff2 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:39.827975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4592c8-da97-4058-9b58-c4be2336fafd None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:39.828504 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4592c8-da97-4058-9b58-c4be2336fafd None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:39.828585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4592c8-da97-4058-9b58-c4be2336fafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:39.850314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-4d4592c8-da97-4058-9b58-c4be2336fafd None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:39.851720 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4d4592c8-da97-4058-9b58-c4be2336fafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:39.934630 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 940/1874] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:39.942924 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bad722-51bf-4346-b80e-034b7ee2d65d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:39.943214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bad722-51bf-4346-b80e-034b7ee2d65d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:39.943492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bad722-51bf-4346-b80e-034b7ee2d65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:39.966150 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e0bad722-51bf-4346-b80e-034b7ee2d65d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:39.967366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e0bad722-51bf-4346-b80e-034b7ee2d65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:39.975938 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 935/1875] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:39.980814 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4592c8-da97-4058-9b58-c4be2336fafd None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:39.991415 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4592c8-da97-4058-9b58-c4be2336fafd None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:39.997424 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2952ca-a542-4c19-9f21-9616543fbb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:39.998105 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2952ca-a542-4c19-9f21-9616543fbb64 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:39.998387 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2952ca-a542-4c19-9f21-9616543fbb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:39.998659 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2952ca-a542-4c19-9f21-9616543fbb64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:39.999367 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2952ca-a542-4c19-9f21-9616543fbb64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:24:39.999676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2952ca-a542-4c19-9f21-9616543fbb64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:40.001284 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2952ca-a542-4c19-9f21-9616543fbb64 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:40.004031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:40.019310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 941/1876] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:24:40.029714 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bad722-51bf-4346-b80e-034b7ee2d65d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:40.039546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bad722-51bf-4346-b80e-034b7ee2d65d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:40.044788 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d3f6c-a633-4b48-83a0-382dec997107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:40.045440 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d3f6c-a633-4b48-83a0-382dec997107 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:40.045716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d3f6c-a633-4b48-83a0-382dec997107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:40.046024 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d3f6c-a633-4b48-83a0-382dec997107 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:40.047205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d3f6c-a633-4b48-83a0-382dec997107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:40.048935 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d3f6c-a633-4b48-83a0-382dec997107 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:40.079168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 936/1877] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:40 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:40.083335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2952ca-a542-4c19-9f21-9616543fbb64 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:40.094143 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2952ca-a542-4c19-9f21-9616543fbb64 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:40.159316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ef753-9dac-4012-ab54-b9851c4e8ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:40.160058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ef753-9dac-4012-ab54-b9851c4e8ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:40.160496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ef753-9dac-4012-ab54-b9851c4e8ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:40.160835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ef753-9dac-4012-ab54-b9851c4e8ce5 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:40.161580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ef753-9dac-4012-ab54-b9851c4e8ce5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:40.163955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ef753-9dac-4012-ab54-b9851c4e8ce5 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:40.191803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 942/1878] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:40 2022] POST /identity/v3/users => generated 283 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:40.196769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d3f6c-a633-4b48-83a0-382dec997107 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:40.208476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d3f6c-a633-4b48-83a0-382dec997107 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:40.215115 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4add68-6386-475c-886c-d3936b9c9875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:40.215912 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4add68-6386-475c-886c-d3936b9c9875 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:40.216187 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4add68-6386-475c-886c-d3936b9c9875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:40.216460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4add68-6386-475c-886c-d3936b9c9875 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:40.217169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4add68-6386-475c-886c-d3936b9c9875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:40.218865 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4add68-6386-475c-886c-d3936b9c9875 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:40.233424 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 937/1879] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:40.234754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ef753-9dac-4012-ab54-b9851c4e8ce5 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:40.248571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ef753-9dac-4012-ab54-b9851c4e8ce5 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:40.253906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bff5dc-2ebc-49e3-94fb-4411d45d384a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:40.255077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bff5dc-2ebc-49e3-94fb-4411d45d384a admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:40.255077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bff5dc-2ebc-49e3-94fb-4411d45d384a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:40.255273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bff5dc-2ebc-49e3-94fb-4411d45d384a admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:40.255971 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bff5dc-2ebc-49e3-94fb-4411d45d384a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:40.257639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bff5dc-2ebc-49e3-94fb-4411d45d384a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:40.277831 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 943/1880] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:24:40.282733 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4add68-6386-475c-886c-d3936b9c9875 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:40.298048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4add68-6386-475c-886c-d3936b9c9875 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:40.304007 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69ed81-d410-44fd-b291-d58c7814c16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:40.304707 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69ed81-d410-44fd-b291-d58c7814c16c admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:40.304997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69ed81-d410-44fd-b291-d58c7814c16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:40.305277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69ed81-d410-44fd-b291-d58c7814c16c admin admin] PATH_INFO: `/v3/projects/f466fa529c814a6dad6db12e57144847/users/44639dcc203d42a2906beb7eee3a593d/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:40.324592 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69ed81-d410-44fd-b291-d58c7814c16c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f466fa529c814a6dad6db12e57144847, user_id=44639dcc203d42a2906beb7eee3a593d, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:40.327476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69ed81-d410-44fd-b291-d58c7814c16c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:40.349474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 938/1881] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:40 2022] PUT /identity/v3/projects/f466fa529c814a6dad6db12e57144847/users/44639dcc203d42a2906beb7eee3a593d/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:40.360850 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fe6eb-5ba7-4757-ad6b-f30fa8bdebb5 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:40.361137 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fe6eb-5ba7-4757-ad6b-f30fa8bdebb5 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:40.361395 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fe6eb-5ba7-4757-ad6b-f30fa8bdebb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:40.400492 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-7f8fe6eb-5ba7-4757-ad6b-f30fa8bdebb5 None None] MFA Rules not processed for user `44639dcc203d42a2906beb7eee3a593d`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:40.401866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-7f8fe6eb-5ba7-4757-ad6b-f30fa8bdebb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:40.475732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 944/1882] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:24:40.480397 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69ed81-d410-44fd-b291-d58c7814c16c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:40.548780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69ed81-d410-44fd-b291-d58c7814c16c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:40.554715 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee42ed-d25c-42ed-bc3b-f07abd4fa05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:40.555540 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee42ed-d25c-42ed-bc3b-f07abd4fa05a admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:40.555701 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee42ed-d25c-42ed-bc3b-f07abd4fa05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:40.555975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee42ed-d25c-42ed-bc3b-f07abd4fa05a admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:40.556645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee42ed-d25c-42ed-bc3b-f07abd4fa05a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:40.558670 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee42ed-d25c-42ed-bc3b-f07abd4fa05a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:40.619656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 939/1883] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:40 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:40.624117 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8fe6eb-5ba7-4757-ad6b-f30fa8bdebb5 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:40.639652 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8fe6eb-5ba7-4757-ad6b-f30fa8bdebb5 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:40.717704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370bc0c-c7e8-4ca0-a100-f488dfff016f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:40.718394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370bc0c-c7e8-4ca0-a100-f488dfff016f admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:40.718673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370bc0c-c7e8-4ca0-a100-f488dfff016f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:40.718942 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370bc0c-c7e8-4ca0-a100-f488dfff016f admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:40.719639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370bc0c-c7e8-4ca0-a100-f488dfff016f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:40.721906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370bc0c-c7e8-4ca0-a100-f488dfff016f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:40.734399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:40.752813 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 945/1884] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:40 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:24:40.759769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee42ed-d25c-42ed-bc3b-f07abd4fa05a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:40.771512 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee42ed-d25c-42ed-bc3b-f07abd4fa05a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:40.776221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd28d8e-7e8d-412c-b4af-ef43396676d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:40.776861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd28d8e-7e8d-412c-b4af-ef43396676d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:40.777137 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd28d8e-7e8d-412c-b4af-ef43396676d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:40.777429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd28d8e-7e8d-412c-b4af-ef43396676d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:40.778165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd28d8e-7e8d-412c-b4af-ef43396676d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:40.779810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd28d8e-7e8d-412c-b4af-ef43396676d7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:40.791596 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 940/1885] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:40.796135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370bc0c-c7e8-4ca0-a100-f488dfff016f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:40.807070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370bc0c-c7e8-4ca0-a100-f488dfff016f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:40.811696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3ce2e-e7d3-44e4-94f7-302815d96e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:40.812384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3ce2e-e7d3-44e4-94f7-302815d96e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:40.812645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3ce2e-e7d3-44e4-94f7-302815d96e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:40.812916 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3ce2e-e7d3-44e4-94f7-302815d96e89 admin admin] PATH_INFO: `/v3/projects/3be6b1c7777b4870becd1a3a81237100/users/78b88fdb32c1459a9181c61313469da3/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:40.830028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3ce2e-e7d3-44e4-94f7-302815d96e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3be6b1c7777b4870becd1a3a81237100, user_id=78b88fdb32c1459a9181c61313469da3, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:40.832912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3ce2e-e7d3-44e4-94f7-302815d96e89 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:40.849117 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 946/1886] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:40 2022] PUT /identity/v3/projects/3be6b1c7777b4870becd1a3a81237100/users/78b88fdb32c1459a9181c61313469da3/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:24:40.853600 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd28d8e-7e8d-412c-b4af-ef43396676d7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:40.859635 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:40.874068 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd28d8e-7e8d-412c-b4af-ef43396676d7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:40.909183 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3ce2e-e7d3-44e4-94f7-302815d96e89 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:40.915908 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-53f3ce2e-e7d3-44e4-94f7-302815d96e89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:41.005081 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbad46-18b0-4b63-8e8e-18f60c27a8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:41.005939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbad46-18b0-4b63-8e8e-18f60c27a8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:41.006272 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbad46-18b0-4b63-8e8e-18f60c27a8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:41.006666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbad46-18b0-4b63-8e8e-18f60c27a8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:41.007523 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbad46-18b0-4b63-8e8e-18f60c27a8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:41.010977 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbad46-18b0-4b63-8e8e-18f60c27a8b2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:41.040725 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 941/1887] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:41.045637 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbad46-18b0-4b63-8e8e-18f60c27a8b2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:41.055509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3ce2e-e7d3-44e4-94f7-302815d96e89 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:41.061842 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd8920-a2f5-4a5f-865d-d8746317ae5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:41.063005 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd8920-a2f5-4a5f-865d-d8746317ae5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:41.063425 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd8920-a2f5-4a5f-865d-d8746317ae5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:41.064193 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd8920-a2f5-4a5f-865d-d8746317ae5a admin admin] PATH_INFO: `/v3/users/0e8811f413d1411da8b19630feeb7cfb` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:41.066016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd8920-a2f5-4a5f-865d-d8746317ae5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e8811f413d1411da8b19630feeb7cfb)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:41.069214 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd8920-a2f5-4a5f-865d-d8746317ae5a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:41.070015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbad46-18b0-4b63-8e8e-18f60c27a8b2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:41.080854 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c15965-4053-46b9-823f-ecaa2029451e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:41.082164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c15965-4053-46b9-823f-ecaa2029451e admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:41.082519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c15965-4053-46b9-823f-ecaa2029451e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:41.082856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c15965-4053-46b9-823f-ecaa2029451e admin admin] PATH_INFO: `/v3/projects/3be6b1c7777b4870becd1a3a81237100/users/78b88fdb32c1459a9181c61313469da3/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:41.085473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c15965-4053-46b9-823f-ecaa2029451e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3be6b1c7777b4870becd1a3a81237100, user_id=78b88fdb32c1459a9181c61313469da3, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:41.089562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c15965-4053-46b9-823f-ecaa2029451e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:41.104638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 942/1888] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:41 2022] PUT /identity/v3/projects/3be6b1c7777b4870becd1a3a81237100/users/78b88fdb32c1459a9181c61313469da3/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:41.116929 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306ad97-c32e-4bb9-abcd-aebb47e562d8 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:41.117256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306ad97-c32e-4bb9-abcd-aebb47e562d8 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:41.117713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306ad97-c32e-4bb9-abcd-aebb47e562d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:41.131847 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-57cd8920-a2f5-4a5f-865d-d8746317ae5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e8811f413d1411da8b19630feeb7cfb'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:41.145225 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-57cd8920-a2f5-4a5f-865d-d8746317ae5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e8811f413d1411da8b19630feeb7cfb'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:41.146300 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-57cd8920-a2f5-4a5f-865d-d8746317ae5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e8811f413d1411da8b19630feeb7cfb'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:41.157906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-57cd8920-a2f5-4a5f-865d-d8746317ae5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e8811f413d1411da8b19630feeb7cfb'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:41.161452 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b306ad97-c32e-4bb9-abcd-aebb47e562d8 None None] MFA Rules not processed for user `78b88fdb32c1459a9181c61313469da3`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:41.163358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b306ad97-c32e-4bb9-abcd-aebb47e562d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:41.178441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-57cd8920-a2f5-4a5f-865d-d8746317ae5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e8811f413d1411da8b19630feeb7cfb'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:41.188213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 947/1889] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:40 2022] DELETE /identity/v3/users/0e8811f413d1411da8b19630feeb7cfb => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:24:41.194875 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1907e-6f47-46b9-8524-b8f12c037b45 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:41.195230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1907e-6f47-46b9-8524-b8f12c037b45 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:41.195567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1907e-6f47-46b9-8524-b8f12c037b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:41.221246 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-0be1907e-6f47-46b9-8524-b8f12c037b45 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:41.222518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-0be1907e-6f47-46b9-8524-b8f12c037b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:41.269537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 943/1890] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:41 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:41.283112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306ad97-c32e-4bb9-abcd-aebb47e562d8 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:41.285085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b306ad97-c32e-4bb9-abcd-aebb47e562d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:41.287283 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:41.340600 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 948/1891] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:41.353335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1907e-6f47-46b9-8524-b8f12c037b45 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:41.354841 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-0be1907e-6f47-46b9-8524-b8f12c037b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:41.428491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306ad97-c32e-4bb9-abcd-aebb47e562d8 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:41.494299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:41.511812 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1907e-6f47-46b9-8524-b8f12c037b45 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:41.521951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6ccba-1795-47f1-8926-be71adccf02d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:41.522610 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6ccba-1795-47f1-8926-be71adccf02d service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:41.522889 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6ccba-1795-47f1-8926-be71adccf02d service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:41.523162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6ccba-1795-47f1-8926-be71adccf02d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:41.543340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4f589-9585-4405-8807-3d48a2d296b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:41.544595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4f589-9585-4405-8807-3d48a2d296b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:41.545829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4f589-9585-4405-8807-3d48a2d296b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:41.547430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4f589-9585-4405-8807-3d48a2d296b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:41.549233 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6ccba-1795-47f1-8926-be71adccf02d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:41.551046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6ccba-1795-47f1-8926-be71adccf02d service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:41.577165 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 949/1892] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:24:41.702080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4f589-9585-4405-8807-3d48a2d296b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:41.704407 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4f589-9585-4405-8807-3d48a2d296b9 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:41.738351 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 944/1893] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:41 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:24:41.931159 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6ccba-1795-47f1-8926-be71adccf02d service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:41.932878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5ef6ccba-1795-47f1-8926-be71adccf02d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:41.954959 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6ccba-1795-47f1-8926-be71adccf02d service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:41.967261 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308d36e-de3b-4e76-a9d5-ec789187e447 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:41.967782 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308d36e-de3b-4e76-a9d5-ec789187e447 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:41.967975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308d36e-de3b-4e76-a9d5-ec789187e447 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:41.968265 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d308d36e-de3b-4e76-a9d5-ec789187e447 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:41.972827 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d308d36e-de3b-4e76-a9d5-ec789187e447 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:42.075252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308d36e-de3b-4e76-a9d5-ec789187e447 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:42.076721 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d308d36e-de3b-4e76-a9d5-ec789187e447 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:42.095761 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 950/1894] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:41 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:24:42.107633 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4f589-9585-4405-8807-3d48a2d296b9 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:42.109148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-34e4f589-9585-4405-8807-3d48a2d296b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:42.136929 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4f589-9585-4405-8807-3d48a2d296b9 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:42.156186 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07308778-17f6-4951-ad8d-49724f762f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:42.156990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07308778-17f6-4951-ad8d-49724f762f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:42.157509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07308778-17f6-4951-ad8d-49724f762f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:42.157867 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07308778-17f6-4951-ad8d-49724f762f12 admin admin] PATH_INFO: `/v3/projects/a75bb40bbb7540bab94903f318b208ee` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:42.159328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07308778-17f6-4951-ad8d-49724f762f12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a75bb40bbb7540bab94903f318b208ee)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:42.164429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07308778-17f6-4951-ad8d-49724f762f12 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:42.221126 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-07308778-17f6-4951-ad8d-49724f762f12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a75bb40bbb7540bab94903f318b208ee', 'request_id': 'req-07308778-17f6-4951-ad8d-49724f762f12'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:42.229847 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-07308778-17f6-4951-ad8d-49724f762f12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a75bb40bbb7540bab94903f318b208ee', 'request_id': 'req-07308778-17f6-4951-ad8d-49724f762f12'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:42.242772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-07308778-17f6-4951-ad8d-49724f762f12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a75bb40bbb7540bab94903f318b208ee', 'request_id': 'req-07308778-17f6-4951-ad8d-49724f762f12'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:42.257367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-07308778-17f6-4951-ad8d-49724f762f12 admin admin] The token cache is being invalidate because project a75bb40bbb7540bab94903f318b208ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:42.257662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-07308778-17f6-4951-ad8d-49724f762f12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:42.259200 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-07308778-17f6-4951-ad8d-49724f762f12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:42.264711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 945/1895] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:42 2022] DELETE /identity/v3/projects/a75bb40bbb7540bab94903f318b208ee => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:42.265503 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308d36e-de3b-4e76-a9d5-ec789187e447 service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:42.278455 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d308d36e-de3b-4e76-a9d5-ec789187e447 service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:42.377134 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6f3d3-95e1-4d8a-aaae-c1e5d8e422cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:42.378054 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6f3d3-95e1-4d8a-aaae-c1e5d8e422cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:42.378179 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6f3d3-95e1-4d8a-aaae-c1e5d8e422cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:42.378453 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6f3d3-95e1-4d8a-aaae-c1e5d8e422cd admin admin] PATH_INFO: `/v3/users/3490b2fcdcbc4ee9a3aaf1feed368457` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:42.380108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6f3d3-95e1-4d8a-aaae-c1e5d8e422cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3490b2fcdcbc4ee9a3aaf1feed368457)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:42.382276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6f3d3-95e1-4d8a-aaae-c1e5d8e422cd admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:42.445551 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:42.473917 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-29c6f3d3-95e1-4d8a-aaae-c1e5d8e422cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3490b2fcdcbc4ee9a3aaf1feed368457'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:42.497848 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-29c6f3d3-95e1-4d8a-aaae-c1e5d8e422cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3490b2fcdcbc4ee9a3aaf1feed368457'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:42.498591 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-29c6f3d3-95e1-4d8a-aaae-c1e5d8e422cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3490b2fcdcbc4ee9a3aaf1feed368457'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:42.510082 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-29c6f3d3-95e1-4d8a-aaae-c1e5d8e422cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3490b2fcdcbc4ee9a3aaf1feed368457'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:42.528409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-29c6f3d3-95e1-4d8a-aaae-c1e5d8e422cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3490b2fcdcbc4ee9a3aaf1feed368457'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:42.535565 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 951/1896] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:42 2022] DELETE /identity/v3/users/3490b2fcdcbc4ee9a3aaf1feed368457 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:42.835772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07308778-17f6-4951-ad8d-49724f762f12 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:42.837772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-07308778-17f6-4951-ad8d-49724f762f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:42.854585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07308778-17f6-4951-ad8d-49724f762f12 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:42.947896 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76aaf04-2a5b-40f6-bd84-45b294130ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:42.948840 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76aaf04-2a5b-40f6-bd84-45b294130ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:42.949495 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76aaf04-2a5b-40f6-bd84-45b294130ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:42.949858 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76aaf04-2a5b-40f6-bd84-45b294130ec6 admin admin] PATH_INFO: `/v3/projects/04efe9f324e44df9b8ff0986f2599fb9` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:42.951094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76aaf04-2a5b-40f6-bd84-45b294130ec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04efe9f324e44df9b8ff0986f2599fb9)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:42.953279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76aaf04-2a5b-40f6-bd84-45b294130ec6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:42.955171 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:43.057576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c76aaf04-2a5b-40f6-bd84-45b294130ec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04efe9f324e44df9b8ff0986f2599fb9', 'request_id': 'req-c76aaf04-2a5b-40f6-bd84-45b294130ec6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:43.063978 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c76aaf04-2a5b-40f6-bd84-45b294130ec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04efe9f324e44df9b8ff0986f2599fb9', 'request_id': 'req-c76aaf04-2a5b-40f6-bd84-45b294130ec6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:43.072910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c76aaf04-2a5b-40f6-bd84-45b294130ec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04efe9f324e44df9b8ff0986f2599fb9', 'request_id': 'req-c76aaf04-2a5b-40f6-bd84-45b294130ec6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:43.101442 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c76aaf04-2a5b-40f6-bd84-45b294130ec6 admin admin] The token cache is being invalidate because project 04efe9f324e44df9b8ff0986f2599fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:43.101801 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c76aaf04-2a5b-40f6-bd84-45b294130ec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:43.103202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c76aaf04-2a5b-40f6-bd84-45b294130ec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:43.108974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 946/1897] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:42 2022] DELETE /identity/v3/projects/04efe9f324e44df9b8ff0986f2599fb9 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:43.110240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6f3d3-95e1-4d8a-aaae-c1e5d8e422cd admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:43.112090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-29c6f3d3-95e1-4d8a-aaae-c1e5d8e422cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:43.127817 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6f3d3-95e1-4d8a-aaae-c1e5d8e422cd admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:43.225467 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4fcae-145b-4fe5-ac03-85d0f6f9acb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:43.226126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4fcae-145b-4fe5-ac03-85d0f6f9acb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:43.226410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4fcae-145b-4fe5-ac03-85d0f6f9acb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:43.226685 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4fcae-145b-4fe5-ac03-85d0f6f9acb3 admin admin] PATH_INFO: `/v3/users/a6c7bf11ee03404598e54b05c2332090` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:43.227996 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4fcae-145b-4fe5-ac03-85d0f6f9acb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6c7bf11ee03404598e54b05c2332090)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:43.229996 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4fcae-145b-4fe5-ac03-85d0f6f9acb3 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:43.283347 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d7e4fcae-145b-4fe5-ac03-85d0f6f9acb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6c7bf11ee03404598e54b05c2332090'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:43.295278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d7e4fcae-145b-4fe5-ac03-85d0f6f9acb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6c7bf11ee03404598e54b05c2332090'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:43.296176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d7e4fcae-145b-4fe5-ac03-85d0f6f9acb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6c7bf11ee03404598e54b05c2332090'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:43.307665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d7e4fcae-145b-4fe5-ac03-85d0f6f9acb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6c7bf11ee03404598e54b05c2332090'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:43.325885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d7e4fcae-145b-4fe5-ac03-85d0f6f9acb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6c7bf11ee03404598e54b05c2332090'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:43.332900 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 952/1898] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:43 2022] DELETE /identity/v3/users/a6c7bf11ee03404598e54b05c2332090 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:43.909720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76aaf04-2a5b-40f6-bd84-45b294130ec6 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:43.911858 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c76aaf04-2a5b-40f6-bd84-45b294130ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:43.935404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76aaf04-2a5b-40f6-bd84-45b294130ec6 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:44.040454 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462eaa4-4d5c-4385-be9a-1161c7718961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:44.041674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462eaa4-4d5c-4385-be9a-1161c7718961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:44.042059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462eaa4-4d5c-4385-be9a-1161c7718961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:44.042354 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462eaa4-4d5c-4385-be9a-1161c7718961 admin admin] PATH_INFO: `/v3/projects/aa50cc8d209c4b9d8fea46a1e23d7605` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:44.046831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462eaa4-4d5c-4385-be9a-1161c7718961 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa50cc8d209c4b9d8fea46a1e23d7605)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:44.050023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462eaa4-4d5c-4385-be9a-1161c7718961 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:44.109998 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8462eaa4-4d5c-4385-be9a-1161c7718961 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa50cc8d209c4b9d8fea46a1e23d7605', 'request_id': 'req-8462eaa4-4d5c-4385-be9a-1161c7718961'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:44.119424 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8462eaa4-4d5c-4385-be9a-1161c7718961 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa50cc8d209c4b9d8fea46a1e23d7605', 'request_id': 'req-8462eaa4-4d5c-4385-be9a-1161c7718961'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:44.133611 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8462eaa4-4d5c-4385-be9a-1161c7718961 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa50cc8d209c4b9d8fea46a1e23d7605', 'request_id': 'req-8462eaa4-4d5c-4385-be9a-1161c7718961'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:44.151565 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8462eaa4-4d5c-4385-be9a-1161c7718961 admin admin] The token cache is being invalidate because project aa50cc8d209c4b9d8fea46a1e23d7605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:44.151888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8462eaa4-4d5c-4385-be9a-1161c7718961 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:44.153425 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8462eaa4-4d5c-4385-be9a-1161c7718961 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:44.167699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 947/1899] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:43 2022] DELETE /identity/v3/projects/aa50cc8d209c4b9d8fea46a1e23d7605 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:44.168391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7abc0f-1b00-4d6c-8224-303c78fc2718 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:44.168845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7abc0f-1b00-4d6c-8224-303c78fc2718 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:44.168961 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7abc0f-1b00-4d6c-8224-303c78fc2718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:44.195278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-4b7abc0f-1b00-4d6c-8224-303c78fc2718 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:44.197158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4b7abc0f-1b00-4d6c-8224-303c78fc2718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:44.233290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:44.285908 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 953/1900] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:24:44.294277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c4949-d8d6-4b71-9f71-928913bf87bd None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:44.294592 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c4949-d8d6-4b71-9f71-928913bf87bd None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:44.294919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c4949-d8d6-4b71-9f71-928913bf87bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:44.316527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-069c4949-d8d6-4b71-9f71-928913bf87bd None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:44.317896 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-069c4949-d8d6-4b71-9f71-928913bf87bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:44.327448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 948/1901] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:44.332111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7abc0f-1b00-4d6c-8224-303c78fc2718 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:44.344446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7abc0f-1b00-4d6c-8224-303c78fc2718 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:44.350795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42820dcf-200a-4d47-b229-39cc61a01266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:44.351467 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42820dcf-200a-4d47-b229-39cc61a01266 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:44.351749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42820dcf-200a-4d47-b229-39cc61a01266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:44.352066 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42820dcf-200a-4d47-b229-39cc61a01266 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:44.352816 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42820dcf-200a-4d47-b229-39cc61a01266 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:24:44.353128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42820dcf-200a-4d47-b229-39cc61a01266 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:44.354857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42820dcf-200a-4d47-b229-39cc61a01266 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:44.374397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 954/1902] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:24:44.377337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c4949-d8d6-4b71-9f71-928913bf87bd None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:44.391954 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c4949-d8d6-4b71-9f71-928913bf87bd None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:44.413304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dadd58-e5eb-4410-aab2-7a63276eb684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:44.414083 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dadd58-e5eb-4410-aab2-7a63276eb684 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:44.414362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dadd58-e5eb-4410-aab2-7a63276eb684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:44.414699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dadd58-e5eb-4410-aab2-7a63276eb684 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:44.415429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dadd58-e5eb-4410-aab2-7a63276eb684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:44.418207 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dadd58-e5eb-4410-aab2-7a63276eb684 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:44.433726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:44.467244 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 949/1903] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:44 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:44.471470 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42820dcf-200a-4d47-b229-39cc61a01266 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:44.481963 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42820dcf-200a-4d47-b229-39cc61a01266 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:44.546059 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae48fc-a258-4dc0-bd15-4b92eb599cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:44.546730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae48fc-a258-4dc0-bd15-4b92eb599cd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:44.547002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae48fc-a258-4dc0-bd15-4b92eb599cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:44.547326 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae48fc-a258-4dc0-bd15-4b92eb599cd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:44.547978 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae48fc-a258-4dc0-bd15-4b92eb599cd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:44.549968 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae48fc-a258-4dc0-bd15-4b92eb599cd9 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:44.573168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 955/1904] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:44 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:44.577529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dadd58-e5eb-4410-aab2-7a63276eb684 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:44.589646 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dadd58-e5eb-4410-aab2-7a63276eb684 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:44.594388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44224fbe-6504-477f-b418-3f7ded71e199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:44.595032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44224fbe-6504-477f-b418-3f7ded71e199 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:44.595312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44224fbe-6504-477f-b418-3f7ded71e199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:44.595584 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44224fbe-6504-477f-b418-3f7ded71e199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:44.596295 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44224fbe-6504-477f-b418-3f7ded71e199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:44.597971 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44224fbe-6504-477f-b418-3f7ded71e199 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:44.608603 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 950/1905] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:44.612990 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae48fc-a258-4dc0-bd15-4b92eb599cd9 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:44.624017 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae48fc-a258-4dc0-bd15-4b92eb599cd9 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:44.629118 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1877d6-56c9-466a-a340-b2fe47fc944e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:44.629750 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1877d6-56c9-466a-a340-b2fe47fc944e admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:44.630061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1877d6-56c9-466a-a340-b2fe47fc944e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:44.630330 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1877d6-56c9-466a-a340-b2fe47fc944e admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:44.631030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1877d6-56c9-466a-a340-b2fe47fc944e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:44.632668 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1877d6-56c9-466a-a340-b2fe47fc944e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:44.644383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 956/1906] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:44.648996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44224fbe-6504-477f-b418-3f7ded71e199 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:44.660197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44224fbe-6504-477f-b418-3f7ded71e199 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:44.665076 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e6137-2fd5-495d-bf08-f049137d5fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:44.665750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e6137-2fd5-495d-bf08-f049137d5fa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:44.666064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e6137-2fd5-495d-bf08-f049137d5fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:44.666335 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e6137-2fd5-495d-bf08-f049137d5fa6 admin admin] PATH_INFO: `/v3/projects/1131ca660746489a8969169a664c0d90/users/7a72015998fd4ee0a3f1bacb699cd9ab/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:44.686295 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e6137-2fd5-495d-bf08-f049137d5fa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1131ca660746489a8969169a664c0d90, user_id=7a72015998fd4ee0a3f1bacb699cd9ab, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:44.688976 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e6137-2fd5-495d-bf08-f049137d5fa6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:44.705313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 951/1907] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:44 2022] PUT /identity/v3/projects/1131ca660746489a8969169a664c0d90/users/7a72015998fd4ee0a3f1bacb699cd9ab/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:44.717336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f2c97-5dcf-454c-a788-b3f5d61bc656 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:44.717618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f2c97-5dcf-454c-a788-b3f5d61bc656 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:44.717921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f2c97-5dcf-454c-a788-b3f5d61bc656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:44.748156 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-126f2c97-5dcf-454c-a788-b3f5d61bc656 None None] MFA Rules not processed for user `7a72015998fd4ee0a3f1bacb699cd9ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:44.749412 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-126f2c97-5dcf-454c-a788-b3f5d61bc656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:44.837497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 957/1908] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:44.847222 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e6137-2fd5-495d-bf08-f049137d5fa6 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:44.849727 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5e2e6137-2fd5-495d-bf08-f049137d5fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:44.936241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e6137-2fd5-495d-bf08-f049137d5fa6 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:44.940519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeea9a6-828c-432d-9662-718346a8d305 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:44.941990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeea9a6-828c-432d-9662-718346a8d305 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:44.942283 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeea9a6-828c-432d-9662-718346a8d305 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:44.942567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeea9a6-828c-432d-9662-718346a8d305 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:44.958774 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeea9a6-828c-432d-9662-718346a8d305 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:44.960671 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeea9a6-828c-432d-9662-718346a8d305 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:44.977047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 952/1909] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:45.004628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f2c97-5dcf-454c-a788-b3f5d61bc656 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:45.032729 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:45.075303 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f2c97-5dcf-454c-a788-b3f5d61bc656 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:45.080404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03376d5-881c-4431-871c-06925d4c316f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:45.081095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03376d5-881c-4431-871c-06925d4c316f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:45.081375 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03376d5-881c-4431-871c-06925d4c316f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:45.081806 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03376d5-881c-4431-871c-06925d4c316f admin admin] PATH_INFO: `/v3/users/7a72015998fd4ee0a3f1bacb699cd9ab` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:45.083087 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03376d5-881c-4431-871c-06925d4c316f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a72015998fd4ee0a3f1bacb699cd9ab)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:45.085087 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03376d5-881c-4431-871c-06925d4c316f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:45.140968 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c03376d5-881c-4431-871c-06925d4c316f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a72015998fd4ee0a3f1bacb699cd9ab'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:45.150833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c03376d5-881c-4431-871c-06925d4c316f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a72015998fd4ee0a3f1bacb699cd9ab'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:45.151610 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c03376d5-881c-4431-871c-06925d4c316f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a72015998fd4ee0a3f1bacb699cd9ab'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:45.159582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c03376d5-881c-4431-871c-06925d4c316f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a72015998fd4ee0a3f1bacb699cd9ab'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:45.181206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c03376d5-881c-4431-871c-06925d4c316f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a72015998fd4ee0a3f1bacb699cd9ab'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:45.191584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 958/1910] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:45 2022] DELETE /identity/v3/users/7a72015998fd4ee0a3f1bacb699cd9ab => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:45.197205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2848e53-bea3-48f3-8e18-3697ae3923c1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:45.197809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2848e53-bea3-48f3-8e18-3697ae3923c1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:45.197948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2848e53-bea3-48f3-8e18-3697ae3923c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:45.218416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b2848e53-bea3-48f3-8e18-3697ae3923c1 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:45.219758 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b2848e53-bea3-48f3-8e18-3697ae3923c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:45.289670 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 953/1911] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:45.306382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03376d5-881c-4431-871c-06925d4c316f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:45.308457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c03376d5-881c-4431-871c-06925d4c316f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:45.392256 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03376d5-881c-4431-871c-06925d4c316f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:45.397588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4514f-d092-4bde-a1fa-6c881d32be77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:45.398287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4514f-d092-4bde-a1fa-6c881d32be77 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:45.398563 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4514f-d092-4bde-a1fa-6c881d32be77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:45.398868 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4514f-d092-4bde-a1fa-6c881d32be77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:45.408895 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4514f-d092-4bde-a1fa-6c881d32be77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:45.410966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4514f-d092-4bde-a1fa-6c881d32be77 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:45.428966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 959/1912] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:46.159884 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2848e53-bea3-48f3-8e18-3697ae3923c1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:46.161672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b2848e53-bea3-48f3-8e18-3697ae3923c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:46.190531 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2848e53-bea3-48f3-8e18-3697ae3923c1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:46.217447 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eb8f8-bf0d-4615-9f99-07042e394763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:46.218187 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eb8f8-bf0d-4615-9f99-07042e394763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:46.218408 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eb8f8-bf0d-4615-9f99-07042e394763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:46.218680 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eb8f8-bf0d-4615-9f99-07042e394763 admin admin] PATH_INFO: `/v3/projects/1131ca660746489a8969169a664c0d90` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:46.220003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eb8f8-bf0d-4615-9f99-07042e394763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1131ca660746489a8969169a664c0d90)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:46.222043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eb8f8-bf0d-4615-9f99-07042e394763 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:46.344455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3d8eb8f8-bf0d-4615-9f99-07042e394763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1131ca660746489a8969169a664c0d90', 'request_id': 'req-3d8eb8f8-bf0d-4615-9f99-07042e394763'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:46.374380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:46.380185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3d8eb8f8-bf0d-4615-9f99-07042e394763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1131ca660746489a8969169a664c0d90', 'request_id': 'req-3d8eb8f8-bf0d-4615-9f99-07042e394763'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:46.401396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3d8eb8f8-bf0d-4615-9f99-07042e394763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1131ca660746489a8969169a664c0d90', 'request_id': 'req-3d8eb8f8-bf0d-4615-9f99-07042e394763'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:46.421536 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3d8eb8f8-bf0d-4615-9f99-07042e394763 admin admin] The token cache is being invalidate because project 1131ca660746489a8969169a664c0d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:46.422321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3d8eb8f8-bf0d-4615-9f99-07042e394763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:46.423389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3d8eb8f8-bf0d-4615-9f99-07042e394763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:46.425868 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 954/1913] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:46 2022] DELETE /identity/v3/projects/1131ca660746489a8969169a664c0d90 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:46.431747 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a8bb3-1b95-4625-9599-c313facbccdf None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:46.432596 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a8bb3-1b95-4625-9599-c313facbccdf None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:46.433014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a8bb3-1b95-4625-9599-c313facbccdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:46.473400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-1a9a8bb3-1b95-4625-9599-c313facbccdf None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:46.475369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1a9a8bb3-1b95-4625-9599-c313facbccdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:46.572616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:46.643435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 960/1914] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:46.644582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9940bf-727b-471f-ac97-0db91f8bfe80 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:46.645120 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9940bf-727b-471f-ac97-0db91f8bfe80 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:46.645731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9940bf-727b-471f-ac97-0db91f8bfe80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:46.701010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-cf9940bf-727b-471f-ac97-0db91f8bfe80 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:46.709511 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cf9940bf-727b-471f-ac97-0db91f8bfe80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:46.734724 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 955/1915] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:46.740222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a8bb3-1b95-4625-9599-c313facbccdf None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:46.758260 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a8bb3-1b95-4625-9599-c313facbccdf None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:46.775716 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cbee4-e1e0-4c9d-9158-3e21f5fccdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:46.776730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cbee4-e1e0-4c9d-9158-3e21f5fccdc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:46.776990 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cbee4-e1e0-4c9d-9158-3e21f5fccdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:46.777261 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cbee4-e1e0-4c9d-9158-3e21f5fccdc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:46.778262 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cbee4-e1e0-4c9d-9158-3e21f5fccdc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:24:46.778794 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cbee4-e1e0-4c9d-9158-3e21f5fccdc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:46.786319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cbee4-e1e0-4c9d-9158-3e21f5fccdc4 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:46.833354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 961/1916] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:24:46.837583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9940bf-727b-471f-ac97-0db91f8bfe80 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:46.856256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9940bf-727b-471f-ac97-0db91f8bfe80 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:46.900959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537c5ae-8336-4be2-89f4-e129d94a62a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:46.903472 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537c5ae-8336-4be2-89f4-e129d94a62a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:46.904109 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537c5ae-8336-4be2-89f4-e129d94a62a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:46.904755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537c5ae-8336-4be2-89f4-e129d94a62a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:46.906067 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537c5ae-8336-4be2-89f4-e129d94a62a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:46.908587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537c5ae-8336-4be2-89f4-e129d94a62a9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:46.950684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 956/1917] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:46 2022] POST /identity/v3/projects => generated 397 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:46.954825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cbee4-e1e0-4c9d-9158-3e21f5fccdc4 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:46.970557 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cbee4-e1e0-4c9d-9158-3e21f5fccdc4 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:47.059504 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40a6ee-81b4-4b5e-b585-d4b3f6ef8a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:47.060594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40a6ee-81b4-4b5e-b585-d4b3f6ef8a2f admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:47.061398 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40a6ee-81b4-4b5e-b585-d4b3f6ef8a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:47.064176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40a6ee-81b4-4b5e-b585-d4b3f6ef8a2f admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:47.067059 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40a6ee-81b4-4b5e-b585-d4b3f6ef8a2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:47.069655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40a6ee-81b4-4b5e-b585-d4b3f6ef8a2f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:47.096581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 962/1918] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:46 2022] POST /identity/v3/users => generated 299 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:47.100874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537c5ae-8336-4be2-89f4-e129d94a62a9 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:47.113123 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537c5ae-8336-4be2-89f4-e129d94a62a9 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:47.118741 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcc494-4c2e-4c1d-8afe-5d715e256487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:47.119678 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcc494-4c2e-4c1d-8afe-5d715e256487 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:47.119953 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcc494-4c2e-4c1d-8afe-5d715e256487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:47.120227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcc494-4c2e-4c1d-8afe-5d715e256487 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:47.121543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcc494-4c2e-4c1d-8afe-5d715e256487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:47.122982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcc494-4c2e-4c1d-8afe-5d715e256487 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:47.138468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 957/1919] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:47.140858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40a6ee-81b4-4b5e-b585-d4b3f6ef8a2f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:47.153078 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40a6ee-81b4-4b5e-b585-d4b3f6ef8a2f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:47.158311 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b19d7-d265-427d-bf39-b630e3d6e4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:47.158950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b19d7-d265-427d-bf39-b630e3d6e4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:47.159232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b19d7-d265-427d-bf39-b630e3d6e4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:47.159532 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b19d7-d265-427d-bf39-b630e3d6e4a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:47.160404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b19d7-d265-427d-bf39-b630e3d6e4a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:47.162978 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b19d7-d265-427d-bf39-b630e3d6e4a2 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:47.176686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 963/1920] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:47.180914 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcc494-4c2e-4c1d-8afe-5d715e256487 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:47.192499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcc494-4c2e-4c1d-8afe-5d715e256487 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:47.198625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52b7b0-f0c5-4728-8b5b-5f898b48d21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:47.199319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52b7b0-f0c5-4728-8b5b-5f898b48d21e admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:47.199595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52b7b0-f0c5-4728-8b5b-5f898b48d21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:47.199894 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52b7b0-f0c5-4728-8b5b-5f898b48d21e admin admin] PATH_INFO: `/v3/projects/733def053a9c4b89a97076cfea1f5df1/users/a2562c1c57754959867fd88843b93662/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:47.218149 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52b7b0-f0c5-4728-8b5b-5f898b48d21e admin admin] RBAC: Authorizing `identity:create_grant(project_id=733def053a9c4b89a97076cfea1f5df1, user_id=a2562c1c57754959867fd88843b93662, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:47.221059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52b7b0-f0c5-4728-8b5b-5f898b48d21e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:47.236809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 958/1921] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:47 2022] PUT /identity/v3/projects/733def053a9c4b89a97076cfea1f5df1/users/a2562c1c57754959867fd88843b93662/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:47.247963 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067f2c7-1417-4f55-b6fc-d5208a4872f6 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:47.248354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067f2c7-1417-4f55-b6fc-d5208a4872f6 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:47.248588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067f2c7-1417-4f55-b6fc-d5208a4872f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:47.296591 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-f067f2c7-1417-4f55-b6fc-d5208a4872f6 None None] MFA Rules not processed for user `a2562c1c57754959867fd88843b93662`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:47.298235 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f067f2c7-1417-4f55-b6fc-d5208a4872f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:47.392147 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 964/1922] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:47.406513 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52b7b0-f0c5-4728-8b5b-5f898b48d21e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:47.409566 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9d52b7b0-f0c5-4728-8b5b-5f898b48d21e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:47.512638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52b7b0-f0c5-4728-8b5b-5f898b48d21e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:47.516470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325d505-db08-4fe1-817e-b5ba6854f477 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:47.517104 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325d505-db08-4fe1-817e-b5ba6854f477 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:47.517397 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325d505-db08-4fe1-817e-b5ba6854f477 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:47.517643 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325d505-db08-4fe1-817e-b5ba6854f477 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:47.527642 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325d505-db08-4fe1-817e-b5ba6854f477 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:47.529136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325d505-db08-4fe1-817e-b5ba6854f477 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:47.541074 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 959/1923] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:47.800461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067f2c7-1417-4f55-b6fc-d5208a4872f6 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:47.802597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f067f2c7-1417-4f55-b6fc-d5208a4872f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:47.883425 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:47.917881 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067f2c7-1417-4f55-b6fc-d5208a4872f6 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:47.926222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a33b4-99af-405c-bb5b-25476eb2d482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:47.926222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a33b4-99af-405c-bb5b-25476eb2d482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:47.926222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a33b4-99af-405c-bb5b-25476eb2d482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:47.926837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a33b4-99af-405c-bb5b-25476eb2d482 admin admin] PATH_INFO: `/v3/users/44639dcc203d42a2906beb7eee3a593d` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:47.927911 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a33b4-99af-405c-bb5b-25476eb2d482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44639dcc203d42a2906beb7eee3a593d)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:47.930197 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a33b4-99af-405c-bb5b-25476eb2d482 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:48.048409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-016a33b4-99af-405c-bb5b-25476eb2d482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44639dcc203d42a2906beb7eee3a593d'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:48.066638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-016a33b4-99af-405c-bb5b-25476eb2d482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44639dcc203d42a2906beb7eee3a593d'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:48.067326 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-016a33b4-99af-405c-bb5b-25476eb2d482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44639dcc203d42a2906beb7eee3a593d'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:48.084575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-016a33b4-99af-405c-bb5b-25476eb2d482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44639dcc203d42a2906beb7eee3a593d'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:48.150762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-016a33b4-99af-405c-bb5b-25476eb2d482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44639dcc203d42a2906beb7eee3a593d'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:48.171148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 965/1924] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:47 2022] DELETE /identity/v3/users/44639dcc203d42a2906beb7eee3a593d => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:48.175887 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45303c8d-4047-48b2-9b55-c2d5eba280a5 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:48.176183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45303c8d-4047-48b2-9b55-c2d5eba280a5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:48.176458 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45303c8d-4047-48b2-9b55-c2d5eba280a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:48.223483 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-45303c8d-4047-48b2-9b55-c2d5eba280a5 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:48.228513 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-45303c8d-4047-48b2-9b55-c2d5eba280a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:48.432694 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 960/1925] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:48.443886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a33b4-99af-405c-bb5b-25476eb2d482 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:48.445434 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-016a33b4-99af-405c-bb5b-25476eb2d482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:48.530399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a33b4-99af-405c-bb5b-25476eb2d482 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:48.535070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a38d7-4345-4135-9999-9e29273a3890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:48.536055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a38d7-4345-4135-9999-9e29273a3890 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:48.536641 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a38d7-4345-4135-9999-9e29273a3890 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:48.537109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753a38d7-4345-4135-9999-9e29273a3890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:48.549349 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a38d7-4345-4135-9999-9e29273a3890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:48.551698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753a38d7-4345-4135-9999-9e29273a3890 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:48.567287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 966/1926] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:24:49.010372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45303c8d-4047-48b2-9b55-c2d5eba280a5 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:49.011954 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-45303c8d-4047-48b2-9b55-c2d5eba280a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:49.025504 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45303c8d-4047-48b2-9b55-c2d5eba280a5 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:49.031527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2234235-d314-455d-8a71-81be46e0b1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:49.032578 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2234235-d314-455d-8a71-81be46e0b1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:49.033085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2234235-d314-455d-8a71-81be46e0b1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:49.033595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2234235-d314-455d-8a71-81be46e0b1e5 admin admin] PATH_INFO: `/v3/projects/f466fa529c814a6dad6db12e57144847` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:49.035220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2234235-d314-455d-8a71-81be46e0b1e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f466fa529c814a6dad6db12e57144847)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:49.037539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2234235-d314-455d-8a71-81be46e0b1e5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:49.062243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:49.107634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f2234235-d314-455d-8a71-81be46e0b1e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f466fa529c814a6dad6db12e57144847', 'request_id': 'req-f2234235-d314-455d-8a71-81be46e0b1e5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:49.115582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f2234235-d314-455d-8a71-81be46e0b1e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f466fa529c814a6dad6db12e57144847', 'request_id': 'req-f2234235-d314-455d-8a71-81be46e0b1e5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:49.125736 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f2234235-d314-455d-8a71-81be46e0b1e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f466fa529c814a6dad6db12e57144847', 'request_id': 'req-f2234235-d314-455d-8a71-81be46e0b1e5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:49.141153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f2234235-d314-455d-8a71-81be46e0b1e5 admin admin] The token cache is being invalidate because project f466fa529c814a6dad6db12e57144847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:49.141839 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f2234235-d314-455d-8a71-81be46e0b1e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:49.142682 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f2234235-d314-455d-8a71-81be46e0b1e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:49.145914 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 961/1927] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:49 2022] DELETE /identity/v3/projects/f466fa529c814a6dad6db12e57144847 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:49.148804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a38d7-4345-4135-9999-9e29273a3890 service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:49.160021 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753a38d7-4345-4135-9999-9e29273a3890 service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:49.240032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663a6a8-8221-4d97-b68c-37725beebcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:49.240811 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663a6a8-8221-4d97-b68c-37725beebcbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:49.240960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663a6a8-8221-4d97-b68c-37725beebcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:49.241238 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663a6a8-8221-4d97-b68c-37725beebcbe admin admin] PATH_INFO: `/v3/users/78b88fdb32c1459a9181c61313469da3` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:49.242614 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663a6a8-8221-4d97-b68c-37725beebcbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=78b88fdb32c1459a9181c61313469da3)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:49.244759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663a6a8-8221-4d97-b68c-37725beebcbe admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:49.250260 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:49.309171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4663a6a8-8221-4d97-b68c-37725beebcbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78b88fdb32c1459a9181c61313469da3'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:49.317306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4663a6a8-8221-4d97-b68c-37725beebcbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78b88fdb32c1459a9181c61313469da3'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:49.317926 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4663a6a8-8221-4d97-b68c-37725beebcbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78b88fdb32c1459a9181c61313469da3'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:49.325419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4663a6a8-8221-4d97-b68c-37725beebcbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78b88fdb32c1459a9181c61313469da3'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:49.337662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4663a6a8-8221-4d97-b68c-37725beebcbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78b88fdb32c1459a9181c61313469da3'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:49.344509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 967/1928] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:49 2022] DELETE /identity/v3/users/78b88fdb32c1459a9181c61313469da3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:49.647031 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2234235-d314-455d-8a71-81be46e0b1e5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:49.649347 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f2234235-d314-455d-8a71-81be46e0b1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:49.675384 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2234235-d314-455d-8a71-81be46e0b1e5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:49.757020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aecbce2-023a-480a-b799-0a5180721923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:49.757741 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecbce2-023a-480a-b799-0a5180721923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:49.758121 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecbce2-023a-480a-b799-0a5180721923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:49.758406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecbce2-023a-480a-b799-0a5180721923 admin admin] PATH_INFO: `/v3/projects/3be6b1c7777b4870becd1a3a81237100` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:49.759704 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecbce2-023a-480a-b799-0a5180721923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3be6b1c7777b4870becd1a3a81237100)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:49.761740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecbce2-023a-480a-b799-0a5180721923 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:49.830533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2aecbce2-023a-480a-b799-0a5180721923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3be6b1c7777b4870becd1a3a81237100', 'request_id': 'req-2aecbce2-023a-480a-b799-0a5180721923'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:49.837610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2aecbce2-023a-480a-b799-0a5180721923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3be6b1c7777b4870becd1a3a81237100', 'request_id': 'req-2aecbce2-023a-480a-b799-0a5180721923'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:49.846036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2aecbce2-023a-480a-b799-0a5180721923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3be6b1c7777b4870becd1a3a81237100', 'request_id': 'req-2aecbce2-023a-480a-b799-0a5180721923'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:49.858378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2aecbce2-023a-480a-b799-0a5180721923 admin admin] The token cache is being invalidate because project 3be6b1c7777b4870becd1a3a81237100 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:49.858653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2aecbce2-023a-480a-b799-0a5180721923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:49.859321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2aecbce2-023a-480a-b799-0a5180721923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:49.861589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 962/1929] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:49 2022] DELETE /identity/v3/projects/3be6b1c7777b4870becd1a3a81237100 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:24:49.867660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c49c6f-d0fc-4f97-b9b6-7e8feb4c7bfd None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:49.867940 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c49c6f-d0fc-4f97-b9b6-7e8feb4c7bfd None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:49.868211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c49c6f-d0fc-4f97-b9b6-7e8feb4c7bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:49.894117 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-38c49c6f-d0fc-4f97-b9b6-7e8feb4c7bfd None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:49.895386 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-38c49c6f-d0fc-4f97-b9b6-7e8feb4c7bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:49.962525 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 968/1930] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:49.971039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a6010-52e2-4148-9a20-37cb47dd17c4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:49.971330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a6010-52e2-4148-9a20-37cb47dd17c4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:49.971605 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a6010-52e2-4148-9a20-37cb47dd17c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:49.993982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-8f4a6010-52e2-4148-9a20-37cb47dd17c4 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:49.995236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8f4a6010-52e2-4148-9a20-37cb47dd17c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:50.003665 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 963/1931] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:50.007906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c49c6f-d0fc-4f97-b9b6-7e8feb4c7bfd None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:50.018764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c49c6f-d0fc-4f97-b9b6-7e8feb4c7bfd None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:50.023751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878d82f-18ba-453e-91cb-7b7a861f33b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:50.024423 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878d82f-18ba-453e-91cb-7b7a861f33b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:50.024704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878d82f-18ba-453e-91cb-7b7a861f33b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:50.025019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878d82f-18ba-453e-91cb-7b7a861f33b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:50.025892 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b878d82f-18ba-453e-91cb-7b7a861f33b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:24:50.026239 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b878d82f-18ba-453e-91cb-7b7a861f33b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:50.027851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b878d82f-18ba-453e-91cb-7b7a861f33b3 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:50.041929 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 969/1932] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:24:50.046556 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a6010-52e2-4148-9a20-37cb47dd17c4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:50.057815 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a6010-52e2-4148-9a20-37cb47dd17c4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:50.062728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b8407-62ac-4b15-9d36-b6db05883993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:50.063368 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b8407-62ac-4b15-9d36-b6db05883993 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:50.063829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b8407-62ac-4b15-9d36-b6db05883993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:50.063958 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b8407-62ac-4b15-9d36-b6db05883993 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:50.064652 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b8407-62ac-4b15-9d36-b6db05883993 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:50.066643 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b8407-62ac-4b15-9d36-b6db05883993 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:50.098856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 964/1933] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:50 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:50.102822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878d82f-18ba-453e-91cb-7b7a861f33b3 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:50.113626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878d82f-18ba-453e-91cb-7b7a861f33b3 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:50.178896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f38a9a-fc0e-49bf-a577-bd183f5fcb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:50.179737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f38a9a-fc0e-49bf-a577-bd183f5fcb39 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:50.180152 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f38a9a-fc0e-49bf-a577-bd183f5fcb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:50.180608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f38a9a-fc0e-49bf-a577-bd183f5fcb39 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:50.181897 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f38a9a-fc0e-49bf-a577-bd183f5fcb39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:50.185033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f38a9a-fc0e-49bf-a577-bd183f5fcb39 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:50.205301 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:50.216590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 970/1934] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:50 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:50.217862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b8407-62ac-4b15-9d36-b6db05883993 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:50.228972 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b8407-62ac-4b15-9d36-b6db05883993 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:50.239396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7177f-9547-4cf5-b0a9-58a19bde7620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:50.240382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7177f-9547-4cf5-b0a9-58a19bde7620 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:50.240761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7177f-9547-4cf5-b0a9-58a19bde7620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:50.241224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7177f-9547-4cf5-b0a9-58a19bde7620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:50.242693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7177f-9547-4cf5-b0a9-58a19bde7620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:50.244357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7177f-9547-4cf5-b0a9-58a19bde7620 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:50.246655 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:50.264054 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 965/1935] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:50.265267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f38a9a-fc0e-49bf-a577-bd183f5fcb39 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:50.285815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f38a9a-fc0e-49bf-a577-bd183f5fcb39 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:50.294162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f90230-951a-45e2-9a26-caa77c8d6c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:50.295072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f90230-951a-45e2-9a26-caa77c8d6c0c admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:50.295361 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f90230-951a-45e2-9a26-caa77c8d6c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:50.295637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f90230-951a-45e2-9a26-caa77c8d6c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:50.297609 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f90230-951a-45e2-9a26-caa77c8d6c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:50.298053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f90230-951a-45e2-9a26-caa77c8d6c0c admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:50.310688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 971/1936] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:50.316672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7177f-9547-4cf5-b0a9-58a19bde7620 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:50.331087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7177f-9547-4cf5-b0a9-58a19bde7620 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:50.341986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c17b20-ba49-41d0-a0d9-8ab0557895fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:50.341986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c17b20-ba49-41d0-a0d9-8ab0557895fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:50.341986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c17b20-ba49-41d0-a0d9-8ab0557895fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:50.341986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c17b20-ba49-41d0-a0d9-8ab0557895fc admin admin] PATH_INFO: `/v3/projects/d46b34ab180144aba475d8826616d452/users/de8c6500cc1e4c1f975accc810557540/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:50.355377 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c17b20-ba49-41d0-a0d9-8ab0557895fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d46b34ab180144aba475d8826616d452, user_id=de8c6500cc1e4c1f975accc810557540, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:50.358100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c17b20-ba49-41d0-a0d9-8ab0557895fc admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:50.373131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 966/1937] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:50 2022] PUT /identity/v3/projects/d46b34ab180144aba475d8826616d452/users/de8c6500cc1e4c1f975accc810557540/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:50.387912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5152c-1aa1-4f7b-b1db-56edeef90458 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:50.389126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5152c-1aa1-4f7b-b1db-56edeef90458 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:50.389493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5152c-1aa1-4f7b-b1db-56edeef90458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:50.427840 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-e9b5152c-1aa1-4f7b-b1db-56edeef90458 None None] MFA Rules not processed for user `de8c6500cc1e4c1f975accc810557540`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:50.429107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e9b5152c-1aa1-4f7b-b1db-56edeef90458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:50.509374 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 972/1938] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:24:50.522677 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c17b20-ba49-41d0-a0d9-8ab0557895fc admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:50.524496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e6c17b20-ba49-41d0-a0d9-8ab0557895fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:50.584503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c17b20-ba49-41d0-a0d9-8ab0557895fc admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:50.589019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1713af-b477-4761-86ec-7707b4f2af89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:50.589649 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1713af-b477-4761-86ec-7707b4f2af89 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:50.589950 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1713af-b477-4761-86ec-7707b4f2af89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:50.590247 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1713af-b477-4761-86ec-7707b4f2af89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:50.599764 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1713af-b477-4761-86ec-7707b4f2af89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:50.601241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1713af-b477-4761-86ec-7707b4f2af89 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:50.610750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 967/1939] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:50.632697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5152c-1aa1-4f7b-b1db-56edeef90458 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:50.709181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5152c-1aa1-4f7b-b1db-56edeef90458 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:50.714882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cb4f5-68b7-485f-a81d-022c9955e07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:50.715677 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cb4f5-68b7-485f-a81d-022c9955e07f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:50.715988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cb4f5-68b7-485f-a81d-022c9955e07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:50.716266 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cb4f5-68b7-485f-a81d-022c9955e07f admin admin] PATH_INFO: `/v3/users/de8c6500cc1e4c1f975accc810557540` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:50.717488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cb4f5-68b7-485f-a81d-022c9955e07f admin admin] RBAC: Authorizing `identity:delete_user(user_id=de8c6500cc1e4c1f975accc810557540)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:50.719490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cb4f5-68b7-485f-a81d-022c9955e07f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:50.773112 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4e9cb4f5-68b7-485f-a81d-022c9955e07f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de8c6500cc1e4c1f975accc810557540'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:50.781179 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4e9cb4f5-68b7-485f-a81d-022c9955e07f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de8c6500cc1e4c1f975accc810557540'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:50.782001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4e9cb4f5-68b7-485f-a81d-022c9955e07f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de8c6500cc1e4c1f975accc810557540'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:50.788998 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4e9cb4f5-68b7-485f-a81d-022c9955e07f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de8c6500cc1e4c1f975accc810557540'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:50.801428 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4e9cb4f5-68b7-485f-a81d-022c9955e07f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de8c6500cc1e4c1f975accc810557540'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:50.802954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:50.809534 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 973/1940] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:24:50 2022] DELETE /identity/v3/users/de8c6500cc1e4c1f975accc810557540 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:50.817620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29adbf-afba-4035-b5f0-724dfc221390 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:50.818114 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29adbf-afba-4035-b5f0-724dfc221390 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:50.818226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29adbf-afba-4035-b5f0-724dfc221390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:50.839752 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5f29adbf-afba-4035-b5f0-724dfc221390 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:50.841395 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5f29adbf-afba-4035-b5f0-724dfc221390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:50.931027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 968/1941] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:50.940811 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cb4f5-68b7-485f-a81d-022c9955e07f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:50.942635 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4e9cb4f5-68b7-485f-a81d-022c9955e07f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:51.006555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cb4f5-68b7-485f-a81d-022c9955e07f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:51.011056 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d7073-3509-431d-a291-de67a5c6780a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:51.011693 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d7073-3509-431d-a291-de67a5c6780a service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:51.012298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d7073-3509-431d-a291-de67a5c6780a service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:51.012298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d7073-3509-431d-a291-de67a5c6780a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:51.021897 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d7073-3509-431d-a291-de67a5c6780a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:51.023408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d7073-3509-431d-a291-de67a5c6780a service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:51.033597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 974/1942] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:51.325255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29adbf-afba-4035-b5f0-724dfc221390 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:51.326723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5f29adbf-afba-4035-b5f0-724dfc221390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:51.339724 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29adbf-afba-4035-b5f0-724dfc221390 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:51.345411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71fdb1-6446-4c49-9b29-e514878ee46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:51.346123 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71fdb1-6446-4c49-9b29-e514878ee46f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:51.346400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71fdb1-6446-4c49-9b29-e514878ee46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:51.346674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71fdb1-6446-4c49-9b29-e514878ee46f admin admin] PATH_INFO: `/v3/projects/d46b34ab180144aba475d8826616d452` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:51.349865 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71fdb1-6446-4c49-9b29-e514878ee46f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d46b34ab180144aba475d8826616d452)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:51.351960 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e71fdb1-6446-4c49-9b29-e514878ee46f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:51.406668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6e71fdb1-6446-4c49-9b29-e514878ee46f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd46b34ab180144aba475d8826616d452', 'request_id': 'req-6e71fdb1-6446-4c49-9b29-e514878ee46f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:51.412726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6e71fdb1-6446-4c49-9b29-e514878ee46f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd46b34ab180144aba475d8826616d452', 'request_id': 'req-6e71fdb1-6446-4c49-9b29-e514878ee46f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:51.425061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6e71fdb1-6446-4c49-9b29-e514878ee46f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd46b34ab180144aba475d8826616d452', 'request_id': 'req-6e71fdb1-6446-4c49-9b29-e514878ee46f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:51.432687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:51.438827 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6e71fdb1-6446-4c49-9b29-e514878ee46f admin admin] The token cache is being invalidate because project d46b34ab180144aba475d8826616d452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:24:51.439248 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6e71fdb1-6446-4c49-9b29-e514878ee46f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:51.440467 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6e71fdb1-6446-4c49-9b29-e514878ee46f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:24:51.442396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 969/1943] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:24:51 2022] DELETE /identity/v3/projects/d46b34ab180144aba475d8826616d452 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:51.447731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b138aa8-6c67-4f74-8f81-371fe3ce4ba8 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:51.448204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b138aa8-6c67-4f74-8f81-371fe3ce4ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:51.448687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b138aa8-6c67-4f74-8f81-371fe3ce4ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:51.469353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-4b138aa8-6c67-4f74-8f81-371fe3ce4ba8 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:51.470589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4b138aa8-6c67-4f74-8f81-371fe3ce4ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:51.538489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 975/1944] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:51.546343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26145b-7a38-4fa9-8b4c-062f098111cb None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:51.546654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26145b-7a38-4fa9-8b4c-062f098111cb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:51.546935 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26145b-7a38-4fa9-8b4c-062f098111cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:51.568241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3b26145b-7a38-4fa9-8b4c-062f098111cb None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:51.569366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3b26145b-7a38-4fa9-8b4c-062f098111cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:51.577614 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 970/1945] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:51.582259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b138aa8-6c67-4f74-8f81-371fe3ce4ba8 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:51.593020 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b138aa8-6c67-4f74-8f81-371fe3ce4ba8 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:51.597636 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8277fd-f9fa-491b-a131-941c81aca812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:51.598308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8277fd-f9fa-491b-a131-941c81aca812 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:51.598584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8277fd-f9fa-491b-a131-941c81aca812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:51.598860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8277fd-f9fa-491b-a131-941c81aca812 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:51.599624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8277fd-f9fa-491b-a131-941c81aca812 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:24:51.599912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8277fd-f9fa-491b-a131-941c81aca812 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:51.603513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8277fd-f9fa-491b-a131-941c81aca812 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:51.615377 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 976/1946] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:24:51.619969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26145b-7a38-4fa9-8b4c-062f098111cb None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:51.629817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26145b-7a38-4fa9-8b4c-062f098111cb None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:51.635444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290213c-71e1-4719-85e4-fa566065601e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:51.636082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290213c-71e1-4719-85e4-fa566065601e admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:51.636360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290213c-71e1-4719-85e4-fa566065601e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:51.636625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290213c-71e1-4719-85e4-fa566065601e admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:51.637299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290213c-71e1-4719-85e4-fa566065601e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:51.639289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290213c-71e1-4719-85e4-fa566065601e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:51.669541 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 971/1947] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:24:51 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:51.673950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8277fd-f9fa-491b-a131-941c81aca812 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:51.685177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8277fd-f9fa-491b-a131-941c81aca812 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:51.758428 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6829b-af9e-4305-97f5-754cc0c980a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:51.759090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6829b-af9e-4305-97f5-754cc0c980a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:51.759378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6829b-af9e-4305-97f5-754cc0c980a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:51.759648 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6829b-af9e-4305-97f5-754cc0c980a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:51.760329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6829b-af9e-4305-97f5-754cc0c980a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:51.762305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6829b-af9e-4305-97f5-754cc0c980a1 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:51.785123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 977/1948] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:24:51 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:24:51.789632 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290213c-71e1-4719-85e4-fa566065601e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:51.802168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290213c-71e1-4719-85e4-fa566065601e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:51.808260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbf838-6724-487d-9afc-b25a04960fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:51.808950 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbf838-6724-487d-9afc-b25a04960fcc admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:51.809276 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbf838-6724-487d-9afc-b25a04960fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:51.809590 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbf838-6724-487d-9afc-b25a04960fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:51.810478 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedbf838-6724-487d-9afc-b25a04960fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:51.812177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedbf838-6724-487d-9afc-b25a04960fcc admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:51.822972 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 972/1949] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:51.827257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6829b-af9e-4305-97f5-754cc0c980a1 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:51.838667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6829b-af9e-4305-97f5-754cc0c980a1 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:51.843669 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253e05e-2d9f-487f-b671-b6b6f0c1c52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:51.844299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253e05e-2d9f-487f-b671-b6b6f0c1c52b admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:51.844578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253e05e-2d9f-487f-b671-b6b6f0c1c52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:51.844854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253e05e-2d9f-487f-b671-b6b6f0c1c52b admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:51.845585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253e05e-2d9f-487f-b671-b6b6f0c1c52b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:51.847584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253e05e-2d9f-487f-b671-b6b6f0c1c52b admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:51.859215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 978/1950] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:24:51.863780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbf838-6724-487d-9afc-b25a04960fcc admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:51.878225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbf838-6724-487d-9afc-b25a04960fcc admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:51.884307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a85168-1525-4cd5-b7d4-5ff9ebb89654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:51.885039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a85168-1525-4cd5-b7d4-5ff9ebb89654 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:51.885318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a85168-1525-4cd5-b7d4-5ff9ebb89654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:51.885593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a85168-1525-4cd5-b7d4-5ff9ebb89654 admin admin] PATH_INFO: `/v3/projects/9d68661ec35b4ddfa46d37aa688b54e8/users/becf60362c81407ba1a9fb6fc01faf91/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:51.908743 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a85168-1525-4cd5-b7d4-5ff9ebb89654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d68661ec35b4ddfa46d37aa688b54e8, user_id=becf60362c81407ba1a9fb6fc01faf91, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:51.913197 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a85168-1525-4cd5-b7d4-5ff9ebb89654 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:51.929095 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 973/1951] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:24:51 2022] PUT /identity/v3/projects/9d68661ec35b4ddfa46d37aa688b54e8/users/becf60362c81407ba1a9fb6fc01faf91/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:24:51.939622 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b3fe9-780f-4a8b-85ae-4cae8b31be06 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:51.940218 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b3fe9-780f-4a8b-85ae-4cae8b31be06 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:51.940555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b3fe9-780f-4a8b-85ae-4cae8b31be06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:51.954737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:51.971567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-507b3fe9-780f-4a8b-85ae-4cae8b31be06 None None] MFA Rules not processed for user `becf60362c81407ba1a9fb6fc01faf91`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:51.972784 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-507b3fe9-780f-4a8b-85ae-4cae8b31be06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:52.050235 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 979/1952] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:24:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:52.062534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a85168-1525-4cd5-b7d4-5ff9ebb89654 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:52.064443 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f4a85168-1525-4cd5-b7d4-5ff9ebb89654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:52.127342 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a85168-1525-4cd5-b7d4-5ff9ebb89654 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:52.131948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c401fb2-3464-4db7-8ead-208003dd32ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:52.132541 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c401fb2-3464-4db7-8ead-208003dd32ff service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:52.132824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c401fb2-3464-4db7-8ead-208003dd32ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:52.133225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c401fb2-3464-4db7-8ead-208003dd32ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:52.142821 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c401fb2-3464-4db7-8ead-208003dd32ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:52.144374 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c401fb2-3464-4db7-8ead-208003dd32ff service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:52.154239 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 974/1953] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:24:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:25:01.426713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b3fe9-780f-4a8b-85ae-4cae8b31be06 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:01.428640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-507b3fe9-780f-4a8b-85ae-4cae8b31be06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:01.521403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b3fe9-780f-4a8b-85ae-4cae8b31be06 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:01.527191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18cf98-50e6-4410-8365-33763e8886a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:01.529118 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18cf98-50e6-4410-8365-33763e8886a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:01.529474 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18cf98-50e6-4410-8365-33763e8886a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:01.529843 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18cf98-50e6-4410-8365-33763e8886a8 admin admin] PATH_INFO: `/v3/users/5f47a5a90a5c4e56ac373388657e5e1f` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:01.531159 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18cf98-50e6-4410-8365-33763e8886a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f47a5a90a5c4e56ac373388657e5e1f)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:01.533565 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18cf98-50e6-4410-8365-33763e8886a8 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:01.598463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-db18cf98-50e6-4410-8365-33763e8886a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f47a5a90a5c4e56ac373388657e5e1f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:01.608811 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-db18cf98-50e6-4410-8365-33763e8886a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f47a5a90a5c4e56ac373388657e5e1f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:01.609893 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-db18cf98-50e6-4410-8365-33763e8886a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f47a5a90a5c4e56ac373388657e5e1f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:01.619526 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-db18cf98-50e6-4410-8365-33763e8886a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f47a5a90a5c4e56ac373388657e5e1f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:01.647497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-db18cf98-50e6-4410-8365-33763e8886a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f47a5a90a5c4e56ac373388657e5e1f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:01.659644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 980/1954] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:25:01 2022] DELETE /identity/v3/users/5f47a5a90a5c4e56ac373388657e5e1f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:01.661239 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21b014-21dc-44d9-bfd5-0ef73ad06030 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:01.661744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21b014-21dc-44d9-bfd5-0ef73ad06030 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:01.662979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21b014-21dc-44d9-bfd5-0ef73ad06030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:01.685480 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-eb21b014-21dc-44d9-bfd5-0ef73ad06030 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:01.687016 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-eb21b014-21dc-44d9-bfd5-0ef73ad06030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:01.766039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 975/1955] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:01.776334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18cf98-50e6-4410-8365-33763e8886a8 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:01.778236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-db18cf98-50e6-4410-8365-33763e8886a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:01.800445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:01.857952 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18cf98-50e6-4410-8365-33763e8886a8 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:01.860589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd332f8-af40-455d-80a5-c76ecc589a06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:01.861794 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd332f8-af40-455d-80a5-c76ecc589a06 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:01.862128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd332f8-af40-455d-80a5-c76ecc589a06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:01.862477 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd332f8-af40-455d-80a5-c76ecc589a06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:01.874418 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd332f8-af40-455d-80a5-c76ecc589a06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:01.875892 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd332f8-af40-455d-80a5-c76ecc589a06 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:01.888202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 981/1956] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:02.275101 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21b014-21dc-44d9-bfd5-0ef73ad06030 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:02.276574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-eb21b014-21dc-44d9-bfd5-0ef73ad06030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:02.292114 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21b014-21dc-44d9-bfd5-0ef73ad06030 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:02.298524 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de564f42-2581-4b68-8472-7cb529766784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:02.299226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de564f42-2581-4b68-8472-7cb529766784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:02.299512 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de564f42-2581-4b68-8472-7cb529766784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:02.299847 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de564f42-2581-4b68-8472-7cb529766784 admin admin] PATH_INFO: `/v3/projects/087f9994cf454b1aad27728873b37a8f` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:02.301040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de564f42-2581-4b68-8472-7cb529766784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=087f9994cf454b1aad27728873b37a8f)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:02.302994 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de564f42-2581-4b68-8472-7cb529766784 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:02.340393 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:02.354855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-de564f42-2581-4b68-8472-7cb529766784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087f9994cf454b1aad27728873b37a8f', 'request_id': 'req-de564f42-2581-4b68-8472-7cb529766784'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.360732 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-de564f42-2581-4b68-8472-7cb529766784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087f9994cf454b1aad27728873b37a8f', 'request_id': 'req-de564f42-2581-4b68-8472-7cb529766784'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.368372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-de564f42-2581-4b68-8472-7cb529766784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087f9994cf454b1aad27728873b37a8f', 'request_id': 'req-de564f42-2581-4b68-8472-7cb529766784'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.381203 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-de564f42-2581-4b68-8472-7cb529766784 admin admin] The token cache is being invalidate because project 087f9994cf454b1aad27728873b37a8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:25:02.381495 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-de564f42-2581-4b68-8472-7cb529766784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.382226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-de564f42-2581-4b68-8472-7cb529766784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.383873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 976/1957] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:25:02 2022] DELETE /identity/v3/projects/087f9994cf454b1aad27728873b37a8f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:02.388267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd332f8-af40-455d-80a5-c76ecc589a06 service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:02.398956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd332f8-af40-455d-80a5-c76ecc589a06 service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:02.464125 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6278b47-a68b-4689-ad71-8033ef51433b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:02.464788 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6278b47-a68b-4689-ad71-8033ef51433b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:02.465065 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6278b47-a68b-4689-ad71-8033ef51433b admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:02.465337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6278b47-a68b-4689-ad71-8033ef51433b admin admin] PATH_INFO: `/v3/users/09c50cffdf694da39d436964b9052439` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:02.466594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6278b47-a68b-4689-ad71-8033ef51433b admin admin] RBAC: Authorizing `identity:delete_user(user_id=09c50cffdf694da39d436964b9052439)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:02.468656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6278b47-a68b-4689-ad71-8033ef51433b admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:02.520119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c6278b47-a68b-4689-ad71-8033ef51433b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09c50cffdf694da39d436964b9052439'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.537728 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c6278b47-a68b-4689-ad71-8033ef51433b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09c50cffdf694da39d436964b9052439'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.538562 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c6278b47-a68b-4689-ad71-8033ef51433b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09c50cffdf694da39d436964b9052439'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.539723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de564f42-2581-4b68-8472-7cb529766784 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:02.541171 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-de564f42-2581-4b68-8472-7cb529766784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:02.548362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c6278b47-a68b-4689-ad71-8033ef51433b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09c50cffdf694da39d436964b9052439'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.556069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de564f42-2581-4b68-8472-7cb529766784 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:02.561275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584b45f-9491-48b3-904a-2d3064c43ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:02.561438 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c6278b47-a68b-4689-ad71-8033ef51433b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09c50cffdf694da39d436964b9052439'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.562016 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584b45f-9491-48b3-904a-2d3064c43ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:02.562294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584b45f-9491-48b3-904a-2d3064c43ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:02.562595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584b45f-9491-48b3-904a-2d3064c43ca2 admin admin] PATH_INFO: `/v3/users/a2562c1c57754959867fd88843b93662` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:02.564000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584b45f-9491-48b3-904a-2d3064c43ca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2562c1c57754959867fd88843b93662)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:02.566075 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3584b45f-9491-48b3-904a-2d3064c43ca2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:02.568076 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 982/1958] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:25:02 2022] DELETE /identity/v3/users/09c50cffdf694da39d436964b9052439 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:02.619386 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3584b45f-9491-48b3-904a-2d3064c43ca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2562c1c57754959867fd88843b93662'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.620464 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3584b45f-9491-48b3-904a-2d3064c43ca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2562c1c57754959867fd88843b93662'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.635950 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3584b45f-9491-48b3-904a-2d3064c43ca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2562c1c57754959867fd88843b93662'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.648606 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3584b45f-9491-48b3-904a-2d3064c43ca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2562c1c57754959867fd88843b93662'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.666845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3584b45f-9491-48b3-904a-2d3064c43ca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2562c1c57754959867fd88843b93662'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.673524 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 977/1959] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:25:02 2022] DELETE /identity/v3/users/a2562c1c57754959867fd88843b93662 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:02.679049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2a217-3dd3-4b10-befc-614aaa30843b None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:02.679335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2a217-3dd3-4b10-befc-614aaa30843b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:02.679643 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb2a217-3dd3-4b10-befc-614aaa30843b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:02.700467 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-6bb2a217-3dd3-4b10-befc-614aaa30843b None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:02.701635 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6bb2a217-3dd3-4b10-befc-614aaa30843b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:02.738484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:02.772767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 983/1960] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:02.782998 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584b45f-9491-48b3-904a-2d3064c43ca2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:02.784548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3584b45f-9491-48b3-904a-2d3064c43ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:02.847436 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584b45f-9491-48b3-904a-2d3064c43ca2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:02.853179 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12abe4a6-b127-40f8-b5a7-723c12913a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:02.854437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abe4a6-b127-40f8-b5a7-723c12913a6c service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:02.854830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abe4a6-b127-40f8-b5a7-723c12913a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:02.855311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12abe4a6-b127-40f8-b5a7-723c12913a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:02.866338 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abe4a6-b127-40f8-b5a7-723c12913a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:02.868228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12abe4a6-b127-40f8-b5a7-723c12913a6c service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:02.869678 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2a217-3dd3-4b10-befc-614aaa30843b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:02.871484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6bb2a217-3dd3-4b10-befc-614aaa30843b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:02.885652 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 978/1961] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:25:02.890478 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb2a217-3dd3-4b10-befc-614aaa30843b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:02.895707 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28cefe-0a5c-4391-a26d-8068b3733f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:02.896593 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28cefe-0a5c-4391-a26d-8068b3733f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:02.897020 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28cefe-0a5c-4391-a26d-8068b3733f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:02.897602 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28cefe-0a5c-4391-a26d-8068b3733f3d admin admin] PATH_INFO: `/v3/projects/d2aff9beb78a4e7cacdcf0e2958ce57c` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:02.899351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28cefe-0a5c-4391-a26d-8068b3733f3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2aff9beb78a4e7cacdcf0e2958ce57c)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:02.902226 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28cefe-0a5c-4391-a26d-8068b3733f3d admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:02.956304 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6b28cefe-0a5c-4391-a26d-8068b3733f3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2aff9beb78a4e7cacdcf0e2958ce57c', 'request_id': 'req-6b28cefe-0a5c-4391-a26d-8068b3733f3d'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.964184 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6b28cefe-0a5c-4391-a26d-8068b3733f3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2aff9beb78a4e7cacdcf0e2958ce57c', 'request_id': 'req-6b28cefe-0a5c-4391-a26d-8068b3733f3d'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.970988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6b28cefe-0a5c-4391-a26d-8068b3733f3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2aff9beb78a4e7cacdcf0e2958ce57c', 'request_id': 'req-6b28cefe-0a5c-4391-a26d-8068b3733f3d'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.982816 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6b28cefe-0a5c-4391-a26d-8068b3733f3d admin admin] The token cache is being invalidate because project d2aff9beb78a4e7cacdcf0e2958ce57c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:25:02.983189 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6b28cefe-0a5c-4391-a26d-8068b3733f3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.984036 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6b28cefe-0a5c-4391-a26d-8068b3733f3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:02.985691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 984/1962] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:25:02 2022] DELETE /identity/v3/projects/d2aff9beb78a4e7cacdcf0e2958ce57c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:02.995149 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7519d4b-3264-4426-b383-a538a4965781 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:02.995617 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7519d4b-3264-4426-b383-a538a4965781 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:02.995900 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7519d4b-3264-4426-b383-a538a4965781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.018106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b7519d4b-3264-4426-b383-a538a4965781 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:03.019412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b7519d4b-3264-4426-b383-a538a4965781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:03.068448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:03.110820 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 979/1963] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:03.121568 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e2b36-8aca-4be7-8dda-1bb7d0b1ae6c None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.121724 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e2b36-8aca-4be7-8dda-1bb7d0b1ae6c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.122089 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e2b36-8aca-4be7-8dda-1bb7d0b1ae6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.149530 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-c15e2b36-8aca-4be7-8dda-1bb7d0b1ae6c None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:03.151702 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c15e2b36-8aca-4be7-8dda-1bb7d0b1ae6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:03.163909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 985/1964] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:03.168302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7519d4b-3264-4426-b383-a538a4965781 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.180070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7519d4b-3264-4426-b383-a538a4965781 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.195654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18c675-6529-4384-a4e6-45f231a5157b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:03.196266 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18c675-6529-4384-a4e6-45f231a5157b admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.197225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18c675-6529-4384-a4e6-45f231a5157b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.197510 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18c675-6529-4384-a4e6-45f231a5157b admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.198384 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18c675-6529-4384-a4e6-45f231a5157b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:25:03.198699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18c675-6529-4384-a4e6-45f231a5157b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:03.200723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18c675-6529-4384-a4e6-45f231a5157b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:03.218347 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 980/1965] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:25:03.222600 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e2b36-8aca-4be7-8dda-1bb7d0b1ae6c None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.233293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e2b36-8aca-4be7-8dda-1bb7d0b1ae6c None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.239494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18c675-6529-4384-a4e6-45f231a5157b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.241514 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-db18c675-6529-4384-a4e6-45f231a5157b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:03.249840 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45d1e5-9237-4025-a9ad-a9744595ed99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:03.254243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45d1e5-9237-4025-a9ad-a9744595ed99 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.254848 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45d1e5-9237-4025-a9ad-a9744595ed99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.255449 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45d1e5-9237-4025-a9ad-a9744595ed99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.255845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18c675-6529-4384-a4e6-45f231a5157b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.256753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa45d1e5-9237-4025-a9ad-a9744595ed99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:03.258995 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa45d1e5-9237-4025-a9ad-a9744595ed99 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:03.261059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618be5b-4157-450f-8b98-f1ea6faed0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:03.261730 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618be5b-4157-450f-8b98-f1ea6faed0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.262044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618be5b-4157-450f-8b98-f1ea6faed0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.262320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618be5b-4157-450f-8b98-f1ea6faed0b4 admin admin] PATH_INFO: `/v3/projects/733def053a9c4b89a97076cfea1f5df1` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.263738 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618be5b-4157-450f-8b98-f1ea6faed0b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=733def053a9c4b89a97076cfea1f5df1)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:03.266607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618be5b-4157-450f-8b98-f1ea6faed0b4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:03.303951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 986/1966] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:25:03 2022] POST /identity/v3/projects => generated 377 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:03.307474 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45d1e5-9237-4025-a9ad-a9744595ed99 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.331213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d618be5b-4157-450f-8b98-f1ea6faed0b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '733def053a9c4b89a97076cfea1f5df1', 'request_id': 'req-d618be5b-4157-450f-8b98-f1ea6faed0b4'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:03.338263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d618be5b-4157-450f-8b98-f1ea6faed0b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '733def053a9c4b89a97076cfea1f5df1', 'request_id': 'req-d618be5b-4157-450f-8b98-f1ea6faed0b4'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:03.346517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d618be5b-4157-450f-8b98-f1ea6faed0b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '733def053a9c4b89a97076cfea1f5df1', 'request_id': 'req-d618be5b-4157-450f-8b98-f1ea6faed0b4'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:03.360014 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d618be5b-4157-450f-8b98-f1ea6faed0b4 admin admin] The token cache is being invalidate because project 733def053a9c4b89a97076cfea1f5df1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:25:03.360315 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d618be5b-4157-450f-8b98-f1ea6faed0b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:03.361173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d618be5b-4157-450f-8b98-f1ea6faed0b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:03.363858 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 981/1967] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:25:03 2022] DELETE /identity/v3/projects/733def053a9c4b89a97076cfea1f5df1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:03.369128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3df723-ab40-4125-bb4e-483169fc2191 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.369443 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3df723-ab40-4125-bb4e-483169fc2191 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.369720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3df723-ab40-4125-bb4e-483169fc2191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.379356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45d1e5-9237-4025-a9ad-a9744595ed99 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.392778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ad3df723-ab40-4125-bb4e-483169fc2191 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:03.394229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ad3df723-ab40-4125-bb4e-483169fc2191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:03.422773 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:03.452832 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ea392-9608-469d-882a-06ea872a82ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:03.453471 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ea392-9608-469d-882a-06ea872a82ef admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.453748 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ea392-9608-469d-882a-06ea872a82ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.454048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ea392-9608-469d-882a-06ea872a82ef admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.454718 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ea392-9608-469d-882a-06ea872a82ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:03.456667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ea392-9608-469d-882a-06ea872a82ef admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:03.465651 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 982/1968] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:03.476452 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c0d3e-df7a-4d80-83cb-0ec27f0bc8f4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.476740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c0d3e-df7a-4d80-83cb-0ec27f0bc8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.477015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c0d3e-df7a-4d80-83cb-0ec27f0bc8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.484992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 987/1969] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:25:03 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:03.486331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ea392-9608-469d-882a-06ea872a82ef admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.489422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-3e6ea392-9608-469d-882a-06ea872a82ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:03.498922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6a4c0d3e-df7a-4d80-83cb-0ec27f0bc8f4 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:03.500487 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6a4c0d3e-df7a-4d80-83cb-0ec27f0bc8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:03.509945 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ea392-9608-469d-882a-06ea872a82ef admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.515318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 983/1970] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:03.521035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c0d3e-df7a-4d80-83cb-0ec27f0bc8f4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.526984 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c84107c-357a-4a17-bfc9-dbb6121dd01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:03.528349 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84107c-357a-4a17-bfc9-dbb6121dd01a admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.528880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84107c-357a-4a17-bfc9-dbb6121dd01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.529148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84107c-357a-4a17-bfc9-dbb6121dd01a admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.530005 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c84107c-357a-4a17-bfc9-dbb6121dd01a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:03.531430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c0d3e-df7a-4d80-83cb-0ec27f0bc8f4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.532275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c84107c-357a-4a17-bfc9-dbb6121dd01a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:03.536791 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b643abc-c3cb-43ce-9a62-37458c9e0363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:03.538240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b643abc-c3cb-43ce-9a62-37458c9e0363 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.538443 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b643abc-c3cb-43ce-9a62-37458c9e0363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.538921 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b643abc-c3cb-43ce-9a62-37458c9e0363 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.539722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b643abc-c3cb-43ce-9a62-37458c9e0363 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:25:03.539722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b643abc-c3cb-43ce-9a62-37458c9e0363 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:03.541659 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b643abc-c3cb-43ce-9a62-37458c9e0363 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:03.550416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 988/1971] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:03.555749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c84107c-357a-4a17-bfc9-dbb6121dd01a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.556274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 984/1972] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:25:03.560655 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b643abc-c3cb-43ce-9a62-37458c9e0363 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.566107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c84107c-357a-4a17-bfc9-dbb6121dd01a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.570952 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5548adb-b56f-46dd-81e7-b9c630db3b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:03.571446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b643abc-c3cb-43ce-9a62-37458c9e0363 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.571560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5548adb-b56f-46dd-81e7-b9c630db3b24 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.571872 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5548adb-b56f-46dd-81e7-b9c630db3b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.572145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5548adb-b56f-46dd-81e7-b9c630db3b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.572846 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5548adb-b56f-46dd-81e7-b9c630db3b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:03.574594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5548adb-b56f-46dd-81e7-b9c630db3b24 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:03.576732 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4452525-740c-42a5-802b-e84ee8bc80fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:03.577387 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4452525-740c-42a5-802b-e84ee8bc80fb admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.577673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4452525-740c-42a5-802b-e84ee8bc80fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.577989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4452525-740c-42a5-802b-e84ee8bc80fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.578662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4452525-740c-42a5-802b-e84ee8bc80fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:03.580602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4452525-740c-42a5-802b-e84ee8bc80fb admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:03.586001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 989/1973] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:03.590293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5548adb-b56f-46dd-81e7-b9c630db3b24 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.600802 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5548adb-b56f-46dd-81e7-b9c630db3b24 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.605643 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca1e72-c38c-40a6-b202-c45fba8a3f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:03.606342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca1e72-c38c-40a6-b202-c45fba8a3f62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.606620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca1e72-c38c-40a6-b202-c45fba8a3f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.606884 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca1e72-c38c-40a6-b202-c45fba8a3f62 admin admin] PATH_INFO: `/v3/projects/1844f6812b714da39aba8ed3563b563c/users/560f1b0322db43daad2f5c2d4d9d8a74/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.610532 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 985/1974] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:25:03 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:25:03.614668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4452525-740c-42a5-802b-e84ee8bc80fb admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.621020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:03.623088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca1e72-c38c-40a6-b202-c45fba8a3f62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1844f6812b714da39aba8ed3563b563c, user_id=560f1b0322db43daad2f5c2d4d9d8a74, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:03.625724 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca1e72-c38c-40a6-b202-c45fba8a3f62 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:03.639866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 990/1975] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:03 2022] PUT /identity/v3/projects/1844f6812b714da39aba8ed3563b563c/users/560f1b0322db43daad2f5c2d4d9d8a74/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:03.650232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbc164-8303-433e-9bff-ca6d2f268adb None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.650627 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbc164-8303-433e-9bff-ca6d2f268adb None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.650924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbc164-8303-433e-9bff-ca6d2f268adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.678404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-41bbc164-8303-433e-9bff-ca6d2f268adb None None] MFA Rules not processed for user `560f1b0322db43daad2f5c2d4d9d8a74`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:03.679689 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-41bbc164-8303-433e-9bff-ca6d2f268adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:03.681180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4452525-740c-42a5-802b-e84ee8bc80fb admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.741063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b6b99-e783-4d0a-9bda-00fda2812d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:03.741661 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b6b99-e783-4d0a-9bda-00fda2812d35 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.742077 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b6b99-e783-4d0a-9bda-00fda2812d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.742360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b6b99-e783-4d0a-9bda-00fda2812d35 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.743058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b6b99-e783-4d0a-9bda-00fda2812d35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:03.744998 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b6b99-e783-4d0a-9bda-00fda2812d35 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:03.751535 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 991/1976] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:03.752882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bbc164-8303-433e-9bff-ca6d2f268adb None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.764043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bbc164-8303-433e-9bff-ca6d2f268adb None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.770351 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 986/1977] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:25:03 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:25:03.770493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfc7d7-e46b-4e6a-8581-12d16d4386cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:03.770493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfc7d7-e46b-4e6a-8581-12d16d4386cc admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.770493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfc7d7-e46b-4e6a-8581-12d16d4386cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.770493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcfc7d7-e46b-4e6a-8581-12d16d4386cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.770984 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcfc7d7-e46b-4e6a-8581-12d16d4386cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:03.771532 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b6b99-e783-4d0a-9bda-00fda2812d35 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.772901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcfc7d7-e46b-4e6a-8581-12d16d4386cc admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:03.781453 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b6b99-e783-4d0a-9bda-00fda2812d35 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.786437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdca1ec-6c16-4159-af5c-f12d1b0daa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:03.787055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdca1ec-6c16-4159-af5c-f12d1b0daa23 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.787335 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdca1ec-6c16-4159-af5c-f12d1b0daa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.787605 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdca1ec-6c16-4159-af5c-f12d1b0daa23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.788309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdca1ec-6c16-4159-af5c-f12d1b0daa23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:03.790039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdca1ec-6c16-4159-af5c-f12d1b0daa23 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:03.801181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 992/1978] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:25:03 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:03.804297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 987/1979] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:03.805181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfc7d7-e46b-4e6a-8581-12d16d4386cc admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.805562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdca1ec-6c16-4159-af5c-f12d1b0daa23 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.816953 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdca1ec-6c16-4159-af5c-f12d1b0daa23 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.858909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:03.874870 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcfc7d7-e46b-4e6a-8581-12d16d4386cc admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.880275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe87ac-762b-495a-8c3a-f8adc50a78ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:03.881068 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe87ac-762b-495a-8c3a-f8adc50a78ce admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.881356 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe87ac-762b-495a-8c3a-f8adc50a78ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.881668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe87ac-762b-495a-8c3a-f8adc50a78ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.883236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe87ac-762b-495a-8c3a-f8adc50a78ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:03.885463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe87ac-762b-495a-8c3a-f8adc50a78ce admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:03.886591 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83caeb97-6de1-4650-beca-b03d1c0e4a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:03.886591 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83caeb97-6de1-4650-beca-b03d1c0e4a3a admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.887268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83caeb97-6de1-4650-beca-b03d1c0e4a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.887542 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83caeb97-6de1-4650-beca-b03d1c0e4a3a admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.888727 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83caeb97-6de1-4650-beca-b03d1c0e4a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:03.890715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83caeb97-6de1-4650-beca-b03d1c0e4a3a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:03.896787 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 988/1980] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:03.901070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe87ac-762b-495a-8c3a-f8adc50a78ce admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.911931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe87ac-762b-495a-8c3a-f8adc50a78ce admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.916809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650eca8f-cf0b-4c03-91c6-300404bafe97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:03.917357 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 993/1981] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:25:03 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:25:03.920432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eca8f-cf0b-4c03-91c6-300404bafe97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.920727 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eca8f-cf0b-4c03-91c6-300404bafe97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.921014 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650eca8f-cf0b-4c03-91c6-300404bafe97 admin admin] PATH_INFO: `/v3/projects/97f413eea3b945cc843040b11d501993/users/0898fda8be3743f093dbb216f21eb931/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.921313 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83caeb97-6de1-4650-beca-b03d1c0e4a3a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.931521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83caeb97-6de1-4650-beca-b03d1c0e4a3a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.936679 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650eca8f-cf0b-4c03-91c6-300404bafe97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f413eea3b945cc843040b11d501993, user_id=0898fda8be3743f093dbb216f21eb931, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:03.936880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c177163-816a-4093-ba69-186a215d9b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:03.937508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c177163-816a-4093-ba69-186a215d9b47 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.937831 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c177163-816a-4093-ba69-186a215d9b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.938110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c177163-816a-4093-ba69-186a215d9b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.938815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c177163-816a-4093-ba69-186a215d9b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:03.939402 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650eca8f-cf0b-4c03-91c6-300404bafe97 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:03.940407 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c177163-816a-4093-ba69-186a215d9b47 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:03.955798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 994/1982] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:03.957548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c177163-816a-4093-ba69-186a215d9b47 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:03.959526 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 989/1983] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:03 2022] PUT /identity/v3/projects/97f413eea3b945cc843040b11d501993/users/0898fda8be3743f093dbb216f21eb931/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:03.966025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824aea65-b904-469b-acc8-e725d9044589 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:03.966314 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824aea65-b904-469b-acc8-e725d9044589 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:03.966589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824aea65-b904-469b-acc8-e725d9044589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:03.968941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c177163-816a-4093-ba69-186a215d9b47 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:03.974087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:03.988353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-824aea65-b904-469b-acc8-e725d9044589 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:03.989557 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-824aea65-b904-469b-acc8-e725d9044589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:04.027648 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97484439-e6b6-4868-9cd9-1b1606967fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:04.028318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97484439-e6b6-4868-9cd9-1b1606967fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:04.028595 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97484439-e6b6-4868-9cd9-1b1606967fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:04.030332 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97484439-e6b6-4868-9cd9-1b1606967fb2 admin admin] PATH_INFO: `/v3/projects/2507789492e04b9b992b19b9be6ca141/users/652f5974c3294d2388d9ccc027267c17/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:04.044679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97484439-e6b6-4868-9cd9-1b1606967fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2507789492e04b9b992b19b9be6ca141, user_id=652f5974c3294d2388d9ccc027267c17, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:04.047433 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97484439-e6b6-4868-9cd9-1b1606967fb2 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:04.051954 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 990/1984] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:04.062555 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824aea65-b904-469b-acc8-e725d9044589 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:04.064055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-824aea65-b904-469b-acc8-e725d9044589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:04.064340 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 995/1985] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:03 2022] PUT /identity/v3/projects/2507789492e04b9b992b19b9be6ca141/users/652f5974c3294d2388d9ccc027267c17/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:04.067961 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97484439-e6b6-4868-9cd9-1b1606967fb2 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:04.123934 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824aea65-b904-469b-acc8-e725d9044589 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:04.134128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97484439-e6b6-4868-9cd9-1b1606967fb2 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:04.138859 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ba6c3-3fdf-466e-95a7-b3636c210342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:04.139509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ba6c3-3fdf-466e-95a7-b3636c210342 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:04.139792 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ba6c3-3fdf-466e-95a7-b3636c210342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:04.140064 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ba6c3-3fdf-466e-95a7-b3636c210342 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:04.140792 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ba6c3-3fdf-466e-95a7-b3636c210342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:04.142447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ba6c3-3fdf-466e-95a7-b3636c210342 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:04.152705 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 996/1986] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:04.156568 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ba6c3-3fdf-466e-95a7-b3636c210342 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:04.166468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ba6c3-3fdf-466e-95a7-b3636c210342 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:04.171133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b8cd2-b514-426d-b08c-9fb4a54121be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:04.171785 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b8cd2-b514-426d-b08c-9fb4a54121be admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:04.172055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b8cd2-b514-426d-b08c-9fb4a54121be admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:04.172411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b8cd2-b514-426d-b08c-9fb4a54121be admin admin] PATH_INFO: `/v3/projects/2507789492e04b9b992b19b9be6ca141/users/652f5974c3294d2388d9ccc027267c17/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:04.174322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b8cd2-b514-426d-b08c-9fb4a54121be admin admin] RBAC: Authorizing `identity:create_grant(project_id=2507789492e04b9b992b19b9be6ca141, user_id=652f5974c3294d2388d9ccc027267c17, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:04.174760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf74344-f13c-4bb8-ad84-302f3bdaf343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:04.175392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf74344-f13c-4bb8-ad84-302f3bdaf343 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:04.175675 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf74344-f13c-4bb8-ad84-302f3bdaf343 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:04.175951 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf74344-f13c-4bb8-ad84-302f3bdaf343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:04.176931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b8cd2-b514-426d-b08c-9fb4a54121be admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:04.185205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf74344-f13c-4bb8-ad84-302f3bdaf343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:04.186676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf74344-f13c-4bb8-ad84-302f3bdaf343 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:04.186888 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 997/1987] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:04 2022] PUT /identity/v3/projects/2507789492e04b9b992b19b9be6ca141/users/652f5974c3294d2388d9ccc027267c17/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:04.197036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 991/1988] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:04.198420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59819edb-e68b-4131-91a8-6d849c7d7e3c None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:04.198700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59819edb-e68b-4131-91a8-6d849c7d7e3c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:04.198968 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59819edb-e68b-4131-91a8-6d849c7d7e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:04.228332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf74344-f13c-4bb8-ad84-302f3bdaf343 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:04.229873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3bf74344-f13c-4bb8-ad84-302f3bdaf343 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:04.235140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-59819edb-e68b-4131-91a8-6d849c7d7e3c None None] MFA Rules not processed for user `652f5974c3294d2388d9ccc027267c17`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:04.236393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-59819edb-e68b-4131-91a8-6d849c7d7e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:04.246401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf74344-f13c-4bb8-ad84-302f3bdaf343 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:04.262131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:04.324241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e5635-56e8-4442-a36d-4f103453039a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:04.324842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e5635-56e8-4442-a36d-4f103453039a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:04.325185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e5635-56e8-4442-a36d-4f103453039a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:04.325385 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e5635-56e8-4442-a36d-4f103453039a admin admin] PATH_INFO: `/v3/users/becf60362c81407ba1a9fb6fc01faf91` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:04.325996 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 998/1989] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:04.326704 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e5635-56e8-4442-a36d-4f103453039a admin admin] RBAC: Authorizing `identity:delete_user(user_id=becf60362c81407ba1a9fb6fc01faf91)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:04.328668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e5635-56e8-4442-a36d-4f103453039a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:04.331658 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:04.338505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59819edb-e68b-4131-91a8-6d849c7d7e3c None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:04.340137 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-59819edb-e68b-4131-91a8-6d849c7d7e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:04.392110 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-496e5635-56e8-4442-a36d-4f103453039a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'becf60362c81407ba1a9fb6fc01faf91'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:04.392937 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-496e5635-56e8-4442-a36d-4f103453039a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'becf60362c81407ba1a9fb6fc01faf91'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:04.408881 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59819edb-e68b-4131-91a8-6d849c7d7e3c None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:04.409286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-496e5635-56e8-4442-a36d-4f103453039a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'becf60362c81407ba1a9fb6fc01faf91'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:04.413315 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ce8d6-0572-4bec-ad3f-1d6e6c02f68c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:04.414198 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ce8d6-0572-4bec-ad3f-1d6e6c02f68c service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:04.414531 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ce8d6-0572-4bec-ad3f-1d6e6c02f68c service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:04.414811 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971ce8d6-0572-4bec-ad3f-1d6e6c02f68c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:04.419823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-496e5635-56e8-4442-a36d-4f103453039a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'becf60362c81407ba1a9fb6fc01faf91'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:04.429428 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-496e5635-56e8-4442-a36d-4f103453039a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'becf60362c81407ba1a9fb6fc01faf91'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:04.439446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 992/1990] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:25:04 2022] DELETE /identity/v3/users/becf60362c81407ba1a9fb6fc01faf91 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:04.441111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54248936-67ab-4555-ad82-0452ed01e514 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:04.441488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54248936-67ab-4555-ad82-0452ed01e514 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:04.441767 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54248936-67ab-4555-ad82-0452ed01e514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:04.463655 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-54248936-67ab-4555-ad82-0452ed01e514 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:04.464988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-54248936-67ab-4555-ad82-0452ed01e514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:04.488506 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ce8d6-0572-4bec-ad3f-1d6e6c02f68c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:04.490109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971ce8d6-0572-4bec-ad3f-1d6e6c02f68c service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:04.501031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 999/1991] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:04.537399 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 993/1992] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:04.547445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ce8d6-0572-4bec-ad3f-1d6e6c02f68c service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:04.550715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-971ce8d6-0572-4bec-ad3f-1d6e6c02f68c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:04.689940 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971ce8d6-0572-4bec-ad3f-1d6e6c02f68c service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:04.737754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:04.763800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db86750-4b39-4dc5-8d1a-e491ae4baeb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:04.764642 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db86750-4b39-4dc5-8d1a-e491ae4baeb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:04.764926 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db86750-4b39-4dc5-8d1a-e491ae4baeb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:04.765814 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db86750-4b39-4dc5-8d1a-e491ae4baeb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:04.781068 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db86750-4b39-4dc5-8d1a-e491ae4baeb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:04.782567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db86750-4b39-4dc5-8d1a-e491ae4baeb8 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:04.794012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1000/1993] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:05.243269 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54248936-67ab-4555-ad82-0452ed01e514 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:05.246354 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-54248936-67ab-4555-ad82-0452ed01e514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:05.272214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54248936-67ab-4555-ad82-0452ed01e514 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:05.279435 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8842ac-d2ee-4bbd-b7c4-140d866575a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:05.280206 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8842ac-d2ee-4bbd-b7c4-140d866575a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:05.280754 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8842ac-d2ee-4bbd-b7c4-140d866575a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:05.281040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8842ac-d2ee-4bbd-b7c4-140d866575a5 admin admin] PATH_INFO: `/v3/projects/9d68661ec35b4ddfa46d37aa688b54e8` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:05.283449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8842ac-d2ee-4bbd-b7c4-140d866575a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d68661ec35b4ddfa46d37aa688b54e8)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:05.284436 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8842ac-d2ee-4bbd-b7c4-140d866575a5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:05.357162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4c8842ac-d2ee-4bbd-b7c4-140d866575a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d68661ec35b4ddfa46d37aa688b54e8', 'request_id': 'req-4c8842ac-d2ee-4bbd-b7c4-140d866575a5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:05.365530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4c8842ac-d2ee-4bbd-b7c4-140d866575a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d68661ec35b4ddfa46d37aa688b54e8', 'request_id': 'req-4c8842ac-d2ee-4bbd-b7c4-140d866575a5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:05.375412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4c8842ac-d2ee-4bbd-b7c4-140d866575a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d68661ec35b4ddfa46d37aa688b54e8', 'request_id': 'req-4c8842ac-d2ee-4bbd-b7c4-140d866575a5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:05.390406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4c8842ac-d2ee-4bbd-b7c4-140d866575a5 admin admin] The token cache is being invalidate because project 9d68661ec35b4ddfa46d37aa688b54e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:25:05.390693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4c8842ac-d2ee-4bbd-b7c4-140d866575a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:05.391354 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4c8842ac-d2ee-4bbd-b7c4-140d866575a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:05.392958 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 994/1994] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:25:05 2022] DELETE /identity/v3/projects/9d68661ec35b4ddfa46d37aa688b54e8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:05.399751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825b032-21fa-46cd-b4a3-8844d8165725 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:05.399751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825b032-21fa-46cd-b4a3-8844d8165725 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:05.399751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825b032-21fa-46cd-b4a3-8844d8165725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:05.421605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-0825b032-21fa-46cd-b4a3-8844d8165725 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:05.422866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-0825b032-21fa-46cd-b4a3-8844d8165725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:05.489530 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1001/1995] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:05.497706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70956f45-70f1-4cef-9416-c00df923aa18 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:05.497999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70956f45-70f1-4cef-9416-c00df923aa18 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:05.498280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70956f45-70f1-4cef-9416-c00df923aa18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:05.507670 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:05.522549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-70956f45-70f1-4cef-9416-c00df923aa18 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:05.524388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-70956f45-70f1-4cef-9416-c00df923aa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:05.532858 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 995/1996] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:05.537738 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825b032-21fa-46cd-b4a3-8844d8165725 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:05.548005 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825b032-21fa-46cd-b4a3-8844d8165725 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:05.553223 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcc3fb-df6b-4ddf-84a7-1548c3768678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:05.553869 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcc3fb-df6b-4ddf-84a7-1548c3768678 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:05.554168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcc3fb-df6b-4ddf-84a7-1548c3768678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:05.554437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcc3fb-df6b-4ddf-84a7-1548c3768678 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:05.555139 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcc3fb-df6b-4ddf-84a7-1548c3768678 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:25:05.555437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcc3fb-df6b-4ddf-84a7-1548c3768678 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:05.557055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcc3fb-df6b-4ddf-84a7-1548c3768678 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:05.571820 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1002/1997] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:25:05.576489 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70956f45-70f1-4cef-9416-c00df923aa18 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:05.586252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70956f45-70f1-4cef-9416-c00df923aa18 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:05.592560 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03fdd85-8bed-41d4-92f4-addb56b9251f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:05.593128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fdd85-8bed-41d4-92f4-addb56b9251f admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:05.593403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fdd85-8bed-41d4-92f4-addb56b9251f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:05.593671 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fdd85-8bed-41d4-92f4-addb56b9251f admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:05.594374 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03fdd85-8bed-41d4-92f4-addb56b9251f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:05.596298 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03fdd85-8bed-41d4-92f4-addb56b9251f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:05.642800 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 996/1998] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:25:05 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:25:05.646912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcc3fb-df6b-4ddf-84a7-1548c3768678 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:05.660921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcc3fb-df6b-4ddf-84a7-1548c3768678 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:05.752639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d7f65-33de-46f5-9d02-35dd26a9a02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:05.753430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d7f65-33de-46f5-9d02-35dd26a9a02a admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:05.753996 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d7f65-33de-46f5-9d02-35dd26a9a02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:05.754280 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d7f65-33de-46f5-9d02-35dd26a9a02a admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:05.757057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d7f65-33de-46f5-9d02-35dd26a9a02a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:05.766166 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d7f65-33de-46f5-9d02-35dd26a9a02a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:05.805838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1003/1999] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:25:05 2022] POST /identity/v3/users => generated 288 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:05.808512 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03fdd85-8bed-41d4-92f4-addb56b9251f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:05.823163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03fdd85-8bed-41d4-92f4-addb56b9251f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:05.836174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0e079-626d-4f8c-b2dc-5d1e48704a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:05.836873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0e079-626d-4f8c-b2dc-5d1e48704a7b admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:05.837216 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0e079-626d-4f8c-b2dc-5d1e48704a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:05.837531 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b0e079-626d-4f8c-b2dc-5d1e48704a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:05.838641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0e079-626d-4f8c-b2dc-5d1e48704a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:05.840879 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b0e079-626d-4f8c-b2dc-5d1e48704a7b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:05.865939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 997/2000] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:25:05.870405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d7f65-33de-46f5-9d02-35dd26a9a02a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:05.893766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d7f65-33de-46f5-9d02-35dd26a9a02a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:05.920099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb99ca-bea8-4447-b94e-715f5ef8d6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:05.921064 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb99ca-bea8-4447-b94e-715f5ef8d6fd admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:05.921360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb99ca-bea8-4447-b94e-715f5ef8d6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:05.921629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb99ca-bea8-4447-b94e-715f5ef8d6fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:05.923992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb99ca-bea8-4447-b94e-715f5ef8d6fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:05.925894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb99ca-bea8-4447-b94e-715f5ef8d6fd admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:05.946138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1004/2001] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:25:05.950722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0e079-626d-4f8c-b2dc-5d1e48704a7b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:05.967898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b0e079-626d-4f8c-b2dc-5d1e48704a7b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:05.994211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1eba3-df8e-4c09-a2fb-36471908da44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:05.996406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1eba3-df8e-4c09-a2fb-36471908da44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:06.004742 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1eba3-df8e-4c09-a2fb-36471908da44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:06.004955 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe1eba3-df8e-4c09-a2fb-36471908da44 admin admin] PATH_INFO: `/v3/projects/4ad608aec8b94f28aba65bfabd8e51e3/users/0b8be63e9d494662b9ca0cabf01e86ad/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:06.031869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1eba3-df8e-4c09-a2fb-36471908da44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ad608aec8b94f28aba65bfabd8e51e3, user_id=0b8be63e9d494662b9ca0cabf01e86ad, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:06.035486 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe1eba3-df8e-4c09-a2fb-36471908da44 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:06.065997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 998/2002] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:05 2022] PUT /identity/v3/projects/4ad608aec8b94f28aba65bfabd8e51e3/users/0b8be63e9d494662b9ca0cabf01e86ad/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:06.069518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffb500-72fc-44ed-8f32-3c22e75e48a6 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:06.069871 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffb500-72fc-44ed-8f32-3c22e75e48a6 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:06.070204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffb500-72fc-44ed-8f32-3c22e75e48a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:06.112648 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-27ffb500-72fc-44ed-8f32-3c22e75e48a6 None None] MFA Rules not processed for user `0b8be63e9d494662b9ca0cabf01e86ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:06.114283 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-27ffb500-72fc-44ed-8f32-3c22e75e48a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:06.180940 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:06.289843 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1005/2003] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:06.305400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1eba3-df8e-4c09-a2fb-36471908da44 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:06.308245 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bfe1eba3-df8e-4c09-a2fb-36471908da44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:06.397659 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe1eba3-df8e-4c09-a2fb-36471908da44 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:06.404500 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acd5a3-f4cf-40a6-b33f-c47cba5e2229 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:06.405339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acd5a3-f4cf-40a6-b33f-c47cba5e2229 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:06.405735 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acd5a3-f4cf-40a6-b33f-c47cba5e2229 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:06.406118 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acd5a3-f4cf-40a6-b33f-c47cba5e2229 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:06.416307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84acd5a3-f4cf-40a6-b33f-c47cba5e2229 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:06.417989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84acd5a3-f4cf-40a6-b33f-c47cba5e2229 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:06.436772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 999/2004] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:09.093582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46acf0-745b-4add-96a1-5c44b19b12ad None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:09.094172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46acf0-745b-4add-96a1-5c44b19b12ad None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:09.094172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46acf0-745b-4add-96a1-5c44b19b12ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:09.125752 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2c46acf0-745b-4add-96a1-5c44b19b12ad None None] MFA Rules not processed for user `0898fda8be3743f093dbb216f21eb931`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:09.127192 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2c46acf0-745b-4add-96a1-5c44b19b12ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:09.204118 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1006/2005] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:09.208598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acd5a3-f4cf-40a6-b33f-c47cba5e2229 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:09.210381 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-84acd5a3-f4cf-40a6-b33f-c47cba5e2229 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:09.292039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acd5a3-f4cf-40a6-b33f-c47cba5e2229 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:09.298332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9985ea-7f50-48f3-8a35-7ec0c399231f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:09.298979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9985ea-7f50-48f3-8a35-7ec0c399231f admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:09.299256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9985ea-7f50-48f3-8a35-7ec0c399231f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:09.299526 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9985ea-7f50-48f3-8a35-7ec0c399231f admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:09.300236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9985ea-7f50-48f3-8a35-7ec0c399231f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:09.302325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9985ea-7f50-48f3-8a35-7ec0c399231f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:09.332817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1000/2006] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:25:09 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:09.338409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46acf0-745b-4add-96a1-5c44b19b12ad None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:09.354576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46acf0-745b-4add-96a1-5c44b19b12ad None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:09.456511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03b7d2-5688-477a-ba57-a62b9061ae0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:09.457108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03b7d2-5688-477a-ba57-a62b9061ae0e admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:09.457388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03b7d2-5688-477a-ba57-a62b9061ae0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:09.457658 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03b7d2-5688-477a-ba57-a62b9061ae0e admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:09.458399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03b7d2-5688-477a-ba57-a62b9061ae0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:09.460521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03b7d2-5688-477a-ba57-a62b9061ae0e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:09.492593 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1007/2007] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:25:09 2022] POST /identity/v3/users => generated 296 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:09.496766 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9985ea-7f50-48f3-8a35-7ec0c399231f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:09.512783 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9985ea-7f50-48f3-8a35-7ec0c399231f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:09.518949 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b2d68-912c-44f4-9961-86c1e337b30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:09.519829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b2d68-912c-44f4-9961-86c1e337b30b admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:09.520128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b2d68-912c-44f4-9961-86c1e337b30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:09.520499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b2d68-912c-44f4-9961-86c1e337b30b admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:09.521316 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b2d68-912c-44f4-9961-86c1e337b30b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:09.523748 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b2d68-912c-44f4-9961-86c1e337b30b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:09.540863 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1001/2008] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:09.547642 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03b7d2-5688-477a-ba57-a62b9061ae0e admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:09.561209 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03b7d2-5688-477a-ba57-a62b9061ae0e admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:09.571303 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ca3f3-2a0c-4660-b329-fe87b104e602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:09.572009 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ca3f3-2a0c-4660-b329-fe87b104e602 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:09.572291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ca3f3-2a0c-4660-b329-fe87b104e602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:09.572991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ca3f3-2a0c-4660-b329-fe87b104e602 admin admin] PATH_INFO: `/v3/projects/f3c0f7029fda4d69b7d5bfd20401e79d/users/452f003eb205486895eb397d5f008d09/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:09.595569 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ca3f3-2a0c-4660-b329-fe87b104e602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3c0f7029fda4d69b7d5bfd20401e79d, user_id=452f003eb205486895eb397d5f008d09, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:09.600148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ca3f3-2a0c-4660-b329-fe87b104e602 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:09.616374 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1008/2009] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:09 2022] PUT /identity/v3/projects/f3c0f7029fda4d69b7d5bfd20401e79d/users/452f003eb205486895eb397d5f008d09/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:09.620594 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b2d68-912c-44f4-9961-86c1e337b30b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:09.631893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b2d68-912c-44f4-9961-86c1e337b30b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:09.647390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:09.719227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a4dd7-28b8-452d-8804-d4580c01dc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:09.719860 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a4dd7-28b8-452d-8804-d4580c01dc06 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:09.720140 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a4dd7-28b8-452d-8804-d4580c01dc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:09.720413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a4dd7-28b8-452d-8804-d4580c01dc06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:09.721111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a4dd7-28b8-452d-8804-d4580c01dc06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:09.722782 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a4dd7-28b8-452d-8804-d4580c01dc06 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:09.733411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1002/2010] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:09.738416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ca3f3-2a0c-4660-b329-fe87b104e602 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:09.749627 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ca3f3-2a0c-4660-b329-fe87b104e602 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:09.755770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c35d1-7f2d-4a0e-9db6-207a7ac1249d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:09.756475 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c35d1-7f2d-4a0e-9db6-207a7ac1249d admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:09.756756 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c35d1-7f2d-4a0e-9db6-207a7ac1249d admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:09.757084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c35d1-7f2d-4a0e-9db6-207a7ac1249d admin admin] PATH_INFO: `/v3/projects/f3c0f7029fda4d69b7d5bfd20401e79d/users/452f003eb205486895eb397d5f008d09/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:09.759321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c35d1-7f2d-4a0e-9db6-207a7ac1249d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3c0f7029fda4d69b7d5bfd20401e79d, user_id=452f003eb205486895eb397d5f008d09, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:09.762432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c35d1-7f2d-4a0e-9db6-207a7ac1249d admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:09.774955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1009/2011] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:09 2022] PUT /identity/v3/projects/f3c0f7029fda4d69b7d5bfd20401e79d/users/452f003eb205486895eb397d5f008d09/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:10.514352 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a4dd7-28b8-452d-8804-d4580c01dc06 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:10.516969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ee8a4dd7-28b8-452d-8804-d4580c01dc06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:10.604729 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a4dd7-28b8-452d-8804-d4580c01dc06 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:10.678085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05133d08-c629-421e-923b-774aab16674b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:10.679669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05133d08-c629-421e-923b-774aab16674b service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:10.680052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05133d08-c629-421e-923b-774aab16674b service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:10.680419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05133d08-c629-421e-923b-774aab16674b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:10.682613 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-05133d08-c629-421e-923b-774aab16674b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:10.799561 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05133d08-c629-421e-923b-774aab16674b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:10.801195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05133d08-c629-421e-923b-774aab16674b service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:10.814672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1003/2012] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:11.167627 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d823a9-27a9-48c9-af19-343b2d169d49 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:11.167627 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d823a9-27a9-48c9-af19-343b2d169d49 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:11.167627 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d823a9-27a9-48c9-af19-343b2d169d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:11.203261 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-f8d823a9-27a9-48c9-af19-343b2d169d49 None None] MFA Rules not processed for user `0b8be63e9d494662b9ca0cabf01e86ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:11.207743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f8d823a9-27a9-48c9-af19-343b2d169d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:11.223274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:11.336406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1010/2013] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:14.841928 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd8eb0-f42c-4b6f-92e4-8c03f03637d7 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:14.842402 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd8eb0-f42c-4b6f-92e4-8c03f03637d7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:14.842481 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd8eb0-f42c-4b6f-92e4-8c03f03637d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:14.946965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-04fd8eb0-f42c-4b6f-92e4-8c03f03637d7 None None] MFA Rules not processed for user `452f003eb205486895eb397d5f008d09`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:14.950410 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-04fd8eb0-f42c-4b6f-92e4-8c03f03637d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:14.966036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:15.074027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1004/2014] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:15.145506 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d823a9-27a9-48c9-af19-343b2d169d49 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:15.147097 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f8d823a9-27a9-48c9-af19-343b2d169d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:15.255187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d823a9-27a9-48c9-af19-343b2d169d49 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:15.264599 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b3416-3fa7-4302-a0d8-8719592bdf0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:15.266481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b3416-3fa7-4302-a0d8-8719592bdf0d service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:15.266763 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b3416-3fa7-4302-a0d8-8719592bdf0d service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:15.267042 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b3416-3fa7-4302-a0d8-8719592bdf0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:15.330905 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b3416-3fa7-4302-a0d8-8719592bdf0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:15.332861 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b3416-3fa7-4302-a0d8-8719592bdf0d service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:15.343735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1011/2015] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:25:25.061571 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=25 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:25.339105 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=76 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:38.985320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd8eb0-f42c-4b6f-92e4-8c03f03637d7 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:38.986962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-04fd8eb0-f42c-4b6f-92e4-8c03f03637d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:39.079888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd8eb0-f42c-4b6f-92e4-8c03f03637d7 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:39.084887 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035dab8f-53ad-45cb-adda-71e7d35bca88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:39.085547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dab8f-53ad-45cb-adda-71e7d35bca88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.085916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dab8f-53ad-45cb-adda-71e7d35bca88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.086163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dab8f-53ad-45cb-adda-71e7d35bca88 admin admin] PATH_INFO: `/v3/users/560f1b0322db43daad2f5c2d4d9d8a74` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.087828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035dab8f-53ad-45cb-adda-71e7d35bca88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=560f1b0322db43daad2f5c2d4d9d8a74)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:39.089968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035dab8f-53ad-45cb-adda-71e7d35bca88 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:39.169227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-035dab8f-53ad-45cb-adda-71e7d35bca88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '560f1b0322db43daad2f5c2d4d9d8a74'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:39.170888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-035dab8f-53ad-45cb-adda-71e7d35bca88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '560f1b0322db43daad2f5c2d4d9d8a74'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:39.189179 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-035dab8f-53ad-45cb-adda-71e7d35bca88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '560f1b0322db43daad2f5c2d4d9d8a74'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:39.201723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-035dab8f-53ad-45cb-adda-71e7d35bca88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '560f1b0322db43daad2f5c2d4d9d8a74'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:39.229213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-035dab8f-53ad-45cb-adda-71e7d35bca88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '560f1b0322db43daad2f5c2d4d9d8a74'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:39.239469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8f49d-90c1-4990-a989-4c07db080264 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.240150 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8f49d-90c1-4990-a989-4c07db080264 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.240341 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8f49d-90c1-4990-a989-4c07db080264 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.241835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1012/2016] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 15:25:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:25:39.259503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1005/2017] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:25:38 2022] DELETE /identity/v3/users/560f1b0322db43daad2f5c2d4d9d8a74 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:39.261382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f29cc0-077c-4383-b274-54ab81abad60 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.261801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f29cc0-077c-4383-b274-54ab81abad60 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.262316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f29cc0-077c-4383-b274-54ab81abad60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.289252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-86f29cc0-077c-4383-b274-54ab81abad60 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:39.291031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-86f29cc0-077c-4383-b274-54ab81abad60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:39.369906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1013/2018] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:39.397233 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035dab8f-53ad-45cb-adda-71e7d35bca88 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:39.398767 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-035dab8f-53ad-45cb-adda-71e7d35bca88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:39.485060 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035dab8f-53ad-45cb-adda-71e7d35bca88 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:39.489069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001881c-4f6f-40c6-803b-24c6d1e43a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:39.489723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001881c-4f6f-40c6-803b-24c6d1e43a0a service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.490105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001881c-4f6f-40c6-803b-24c6d1e43a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.490427 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001881c-4f6f-40c6-803b-24c6d1e43a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.500922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001881c-4f6f-40c6-803b-24c6d1e43a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:39.502386 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001881c-4f6f-40c6-803b-24c6d1e43a0a service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:39.509974 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bf197-a4f2-4bd2-a054-fbfe0cb160a5 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.511072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bf197-a4f2-4bd2-a054-fbfe0cb160a5 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.511523 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bf197-a4f2-4bd2-a054-fbfe0cb160a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.515742 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1006/2019] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:25:39.549064 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-169bf197-a4f2-4bd2-a054-fbfe0cb160a5 None None] MFA Rules not processed for user `098d8d7e35804ccc931ddfba07f688a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:39.551443 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-169bf197-a4f2-4bd2-a054-fbfe0cb160a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:39.567260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001881c-4f6f-40c6-803b-24c6d1e43a0a service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:39.569188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7001881c-4f6f-40c6-803b-24c6d1e43a0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:39.590022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001881c-4f6f-40c6-803b-24c6d1e43a0a service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:39.596081 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb7b24-b3bb-40d5-ac34-63207a448c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:39.597072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb7b24-b3bb-40d5-ac34-63207a448c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.597468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb7b24-b3bb-40d5-ac34-63207a448c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.597912 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb7b24-b3bb-40d5-ac34-63207a448c43 admin admin] PATH_INFO: `/v3/users/e6945b8e606f40f1b79bd4aab96a7f1a` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.600943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb7b24-b3bb-40d5-ac34-63207a448c43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6945b8e606f40f1b79bd4aab96a7f1a)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:39.603273 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb7b24-b3bb-40d5-ac34-63207a448c43 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:39.632278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:39.681593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1cdb7b24-b3bb-40d5-ac34-63207a448c43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6945b8e606f40f1b79bd4aab96a7f1a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:39.682641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1cdb7b24-b3bb-40d5-ac34-63207a448c43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6945b8e606f40f1b79bd4aab96a7f1a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:39.685292 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1014/2020] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 15:25:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:39.693633 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bf197-a4f2-4bd2-a054-fbfe0cb160a5 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:39.695481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-169bf197-a4f2-4bd2-a054-fbfe0cb160a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:39.701177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1cdb7b24-b3bb-40d5-ac34-63207a448c43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6945b8e606f40f1b79bd4aab96a7f1a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:39.713756 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bf197-a4f2-4bd2-a054-fbfe0cb160a5 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:39.717124 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1cdb7b24-b3bb-40d5-ac34-63207a448c43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6945b8e606f40f1b79bd4aab96a7f1a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:39.724299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511e737-5330-4d06-b386-03d234f3ab97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '098d8d7e35804ccc931ddfba07f688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:39.725360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511e737-5330-4d06-b386-03d234f3ab97 service placement] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.725835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511e737-5330-4d06-b386-03d234f3ab97 service placement] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.726293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511e737-5330-4d06-b386-03d234f3ab97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.734384 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1cdb7b24-b3bb-40d5-ac34-63207a448c43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6945b8e606f40f1b79bd4aab96a7f1a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:39.738455 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511e737-5330-4d06-b386-03d234f3ab97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:39.740463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511e737-5330-4d06-b386-03d234f3ab97 service placement] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:39.749367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1007/2021] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:25:39 2022] DELETE /identity/v3/users/e6945b8e606f40f1b79bd4aab96a7f1a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:39.757036 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1015/2022] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:25:39.875006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb7b24-b3bb-40d5-ac34-63207a448c43 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:39.880509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1cdb7b24-b3bb-40d5-ac34-63207a448c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:39.942337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511e737-5330-4d06-b386-03d234f3ab97 service placement] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:39.957058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511e737-5330-4d06-b386-03d234f3ab97 service placement] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:39.973059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb7b24-b3bb-40d5-ac34-63207a448c43 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:39.977738 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee4401-5510-4239-b33c-f9df99976330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:39.978436 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee4401-5510-4239-b33c-f9df99976330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:39.978714 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee4401-5510-4239-b33c-f9df99976330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:39.978989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee4401-5510-4239-b33c-f9df99976330 admin admin] PATH_INFO: `/v3/projects/1844f6812b714da39aba8ed3563b563c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:39.980167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee4401-5510-4239-b33c-f9df99976330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1844f6812b714da39aba8ed3563b563c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:39.982149 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee4401-5510-4239-b33c-f9df99976330 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:40.026690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e75fba-572e-4cc9-8d97-e5f3e14c802a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:40.027360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e75fba-572e-4cc9-8d97-e5f3e14c802a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:40.028353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e75fba-572e-4cc9-8d97-e5f3e14c802a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:40.028632 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e75fba-572e-4cc9-8d97-e5f3e14c802a admin admin] PATH_INFO: `/v3/projects/b13cfc3643554bc785b182b3699058e2` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:40.030269 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e75fba-572e-4cc9-8d97-e5f3e14c802a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b13cfc3643554bc785b182b3699058e2)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:40.033008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e75fba-572e-4cc9-8d97-e5f3e14c802a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:40.038586 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-59ee4401-5510-4239-b33c-f9df99976330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1844f6812b714da39aba8ed3563b563c', 'request_id': 'req-59ee4401-5510-4239-b33c-f9df99976330'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.045985 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-59ee4401-5510-4239-b33c-f9df99976330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1844f6812b714da39aba8ed3563b563c', 'request_id': 'req-59ee4401-5510-4239-b33c-f9df99976330'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.063955 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-59ee4401-5510-4239-b33c-f9df99976330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1844f6812b714da39aba8ed3563b563c', 'request_id': 'req-59ee4401-5510-4239-b33c-f9df99976330'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.069390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:40.076679 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-59ee4401-5510-4239-b33c-f9df99976330 admin admin] The token cache is being invalidate because project 1844f6812b714da39aba8ed3563b563c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:25:40.076998 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-59ee4401-5510-4239-b33c-f9df99976330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.077651 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-59ee4401-5510-4239-b33c-f9df99976330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.079411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1008/2023] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:25:39 2022] DELETE /identity/v3/projects/1844f6812b714da39aba8ed3563b563c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:40.083225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee4401-5510-4239-b33c-f9df99976330 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:40.084684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-59ee4401-5510-4239-b33c-f9df99976330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:40.088251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-95e75fba-572e-4cc9-8d97-e5f3e14c802a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b13cfc3643554bc785b182b3699058e2', 'request_id': 'req-95e75fba-572e-4cc9-8d97-e5f3e14c802a'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.097319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-95e75fba-572e-4cc9-8d97-e5f3e14c802a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b13cfc3643554bc785b182b3699058e2', 'request_id': 'req-95e75fba-572e-4cc9-8d97-e5f3e14c802a'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.104472 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-95e75fba-572e-4cc9-8d97-e5f3e14c802a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b13cfc3643554bc785b182b3699058e2', 'request_id': 'req-95e75fba-572e-4cc9-8d97-e5f3e14c802a'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.114278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:40.116790 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-95e75fba-572e-4cc9-8d97-e5f3e14c802a admin admin] The token cache is being invalidate because project b13cfc3643554bc785b182b3699058e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:25:40.117035 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-95e75fba-572e-4cc9-8d97-e5f3e14c802a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.117679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-95e75fba-572e-4cc9-8d97-e5f3e14c802a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.123000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1016/2024] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:25:39 2022] DELETE /identity/v3/projects/b13cfc3643554bc785b182b3699058e2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:40.125200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45205a88-dff3-4c87-8fc9-f9e3893d1b17 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:40.125520 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45205a88-dff3-4c87-8fc9-f9e3893d1b17 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:40.125975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45205a88-dff3-4c87-8fc9-f9e3893d1b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:40.146220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-45205a88-dff3-4c87-8fc9-f9e3893d1b17 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:40.147466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-45205a88-dff3-4c87-8fc9-f9e3893d1b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:40.154598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee4401-5510-4239-b33c-f9df99976330 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:40.208069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1017/2025] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:40.214219 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19937f-8f98-4b8a-b038-a9f465bc5774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:40.214882 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19937f-8f98-4b8a-b038-a9f465bc5774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:40.215162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19937f-8f98-4b8a-b038-a9f465bc5774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:40.215438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19937f-8f98-4b8a-b038-a9f465bc5774 admin admin] PATH_INFO: `/v3/users/652f5974c3294d2388d9ccc027267c17` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:40.216191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435ec5d-8281-4268-a14d-059166d029fa None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:40.216472 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435ec5d-8281-4268-a14d-059166d029fa None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:40.216657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19937f-8f98-4b8a-b038-a9f465bc5774 admin admin] RBAC: Authorizing `identity:delete_user(user_id=652f5974c3294d2388d9ccc027267c17)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:40.216803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435ec5d-8281-4268-a14d-059166d029fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:40.218660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19937f-8f98-4b8a-b038-a9f465bc5774 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:40.237056 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-9435ec5d-8281-4268-a14d-059166d029fa None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:40.238300 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9435ec5d-8281-4268-a14d-059166d029fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:40.246934 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1018/2026] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:40.251473 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435ec5d-8281-4268-a14d-059166d029fa None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:40.261080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435ec5d-8281-4268-a14d-059166d029fa None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:40.265747 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92ea5c-cafc-4d84-943e-9cc2f48a88d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:40.266425 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92ea5c-cafc-4d84-943e-9cc2f48a88d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:40.266707 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92ea5c-cafc-4d84-943e-9cc2f48a88d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:40.266976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92ea5c-cafc-4d84-943e-9cc2f48a88d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:40.267680 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92ea5c-cafc-4d84-943e-9cc2f48a88d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:25:40.267988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92ea5c-cafc-4d84-943e-9cc2f48a88d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:40.268738 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1a19937f-8f98-4b8a-b038-a9f465bc5774 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '652f5974c3294d2388d9ccc027267c17'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.269396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1a19937f-8f98-4b8a-b038-a9f465bc5774 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '652f5974c3294d2388d9ccc027267c17'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.269596 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a92ea5c-cafc-4d84-943e-9cc2f48a88d7 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:40.281346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1019/2027] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:25:40.281493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1a19937f-8f98-4b8a-b038-a9f465bc5774 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '652f5974c3294d2388d9ccc027267c17'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.285910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92ea5c-cafc-4d84-943e-9cc2f48a88d7 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:40.290428 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1a19937f-8f98-4b8a-b038-a9f465bc5774 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '652f5974c3294d2388d9ccc027267c17'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.296019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92ea5c-cafc-4d84-943e-9cc2f48a88d7 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:40.297915 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1a19937f-8f98-4b8a-b038-a9f465bc5774 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '652f5974c3294d2388d9ccc027267c17'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.300588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb76d99-a5e2-4fcd-a546-b0959a3f0031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:40.301193 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb76d99-a5e2-4fcd-a546-b0959a3f0031 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:40.301488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb76d99-a5e2-4fcd-a546-b0959a3f0031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:40.301741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb76d99-a5e2-4fcd-a546-b0959a3f0031 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:40.302424 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb76d99-a5e2-4fcd-a546-b0959a3f0031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:40.303793 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1009/2028] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:25:40 2022] DELETE /identity/v3/users/652f5974c3294d2388d9ccc027267c17 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:40.304374 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb76d99-a5e2-4fcd-a546-b0959a3f0031 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:40.333842 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1020/2029] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:25:40 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:40.337864 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19937f-8f98-4b8a-b038-a9f465bc5774 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:40.339283 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1a19937f-8f98-4b8a-b038-a9f465bc5774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:40.394417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:40.410767 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19937f-8f98-4b8a-b038-a9f465bc5774 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:40.490868 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2407e7-6684-42af-aff2-d2ffc87b51e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:40.491717 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2407e7-6684-42af-aff2-d2ffc87b51e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:40.492042 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2407e7-6684-42af-aff2-d2ffc87b51e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:40.492450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2407e7-6684-42af-aff2-d2ffc87b51e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:40.493280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2407e7-6684-42af-aff2-d2ffc87b51e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:40.495529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2407e7-6684-42af-aff2-d2ffc87b51e6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:40.530047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1010/2030] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:25:40 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:25:40.533681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb76d99-a5e2-4fcd-a546-b0959a3f0031 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:40.544602 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb76d99-a5e2-4fcd-a546-b0959a3f0031 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:40.549896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3424561-7c1a-4a2e-9a62-f2400cc62553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:40.550530 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3424561-7c1a-4a2e-9a62-f2400cc62553 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:40.550808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3424561-7c1a-4a2e-9a62-f2400cc62553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:40.551088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3424561-7c1a-4a2e-9a62-f2400cc62553 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:40.551800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3424561-7c1a-4a2e-9a62-f2400cc62553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:40.553515 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3424561-7c1a-4a2e-9a62-f2400cc62553 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:40.564697 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1021/2031] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:25:40.568689 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2407e7-6684-42af-aff2-d2ffc87b51e6 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:40.579802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2407e7-6684-42af-aff2-d2ffc87b51e6 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:40.586386 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962dc49-3629-4e0a-90d0-35688e578a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:40.587320 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962dc49-3629-4e0a-90d0-35688e578a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:40.587598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962dc49-3629-4e0a-90d0-35688e578a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:40.587870 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962dc49-3629-4e0a-90d0-35688e578a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:40.588574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962dc49-3629-4e0a-90d0-35688e578a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:40.590347 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962dc49-3629-4e0a-90d0-35688e578a17 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:40.604643 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1011/2032] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:40.605731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3424561-7c1a-4a2e-9a62-f2400cc62553 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:40.616429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3424561-7c1a-4a2e-9a62-f2400cc62553 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:40.621427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb921d9-9211-49c1-bafd-2603d80eaf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:40.622146 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb921d9-9211-49c1-bafd-2603d80eaf09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:40.622419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb921d9-9211-49c1-bafd-2603d80eaf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:40.622678 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb921d9-9211-49c1-bafd-2603d80eaf09 admin admin] PATH_INFO: `/v3/projects/3efda1dbd08544eda0f5cea5e3dbe356/users/f9c2d238ddb64745addce164c10f8a40/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:40.640012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb921d9-9211-49c1-bafd-2603d80eaf09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3efda1dbd08544eda0f5cea5e3dbe356, user_id=f9c2d238ddb64745addce164c10f8a40, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:40.642856 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb921d9-9211-49c1-bafd-2603d80eaf09 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:40.659433 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1022/2033] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:40 2022] PUT /identity/v3/projects/3efda1dbd08544eda0f5cea5e3dbe356/users/f9c2d238ddb64745addce164c10f8a40/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:40.669759 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007738-575a-43af-b161-4f66d542392c None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:40.670085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007738-575a-43af-b161-4f66d542392c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:40.670364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007738-575a-43af-b161-4f66d542392c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:40.699662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b6007738-575a-43af-b161-4f66d542392c None None] MFA Rules not processed for user `f9c2d238ddb64745addce164c10f8a40`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:40.700913 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b6007738-575a-43af-b161-4f66d542392c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:40.791667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb921d9-9211-49c1-bafd-2603d80eaf09 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:40.793489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9cb921d9-9211-49c1-bafd-2603d80eaf09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:40.802265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1012/2034] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:40.807006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6007738-575a-43af-b161-4f66d542392c None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:40.862587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:40.890354 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6007738-575a-43af-b161-4f66d542392c None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:40.895195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7f0a8-57c0-4f08-85b0-0c368f486e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:40.895817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7f0a8-57c0-4f08-85b0-0c368f486e88 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:40.895991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb921d9-9211-49c1-bafd-2603d80eaf09 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:40.896117 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7f0a8-57c0-4f08-85b0-0c368f486e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:40.896398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7f0a8-57c0-4f08-85b0-0c368f486e88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:40.897055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7f0a8-57c0-4f08-85b0-0c368f486e88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:40.899017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7f0a8-57c0-4f08-85b0-0c368f486e88 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:40.901185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3955d75-015c-44b7-8991-b969a71939b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:40.901851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3955d75-015c-44b7-8991-b969a71939b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:40.902131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3955d75-015c-44b7-8991-b969a71939b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:40.902401 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3955d75-015c-44b7-8991-b969a71939b2 admin admin] PATH_INFO: `/v3/projects/2507789492e04b9b992b19b9be6ca141` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:40.903566 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3955d75-015c-44b7-8991-b969a71939b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2507789492e04b9b992b19b9be6ca141)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:40.905497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3955d75-015c-44b7-8991-b969a71939b2 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:40.912816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:40.937661 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1013/2035] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:25:40 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:40.938676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7f0a8-57c0-4f08-85b0-0c368f486e88 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:40.962540 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a3955d75-015c-44b7-8991-b969a71939b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2507789492e04b9b992b19b9be6ca141', 'request_id': 'req-a3955d75-015c-44b7-8991-b969a71939b2'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.971059 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a3955d75-015c-44b7-8991-b969a71939b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2507789492e04b9b992b19b9be6ca141', 'request_id': 'req-a3955d75-015c-44b7-8991-b969a71939b2'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.977647 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a3955d75-015c-44b7-8991-b969a71939b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2507789492e04b9b992b19b9be6ca141', 'request_id': 'req-a3955d75-015c-44b7-8991-b969a71939b2'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.989731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a3955d75-015c-44b7-8991-b969a71939b2 admin admin] The token cache is being invalidate because project 2507789492e04b9b992b19b9be6ca141 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:25:40.990095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a3955d75-015c-44b7-8991-b969a71939b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.990877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a3955d75-015c-44b7-8991-b969a71939b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:40.992776 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1023/2036] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:25:40 2022] DELETE /identity/v3/projects/2507789492e04b9b992b19b9be6ca141 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:40.997708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319c365-8692-480c-ba58-946afb06bdea None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:40.998371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319c365-8692-480c-ba58-946afb06bdea None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:40.998831 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319c365-8692-480c-ba58-946afb06bdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.006449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7f0a8-57c0-4f08-85b0-0c368f486e88 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.019794 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-e319c365-8692-480c-ba58-946afb06bdea None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:41.021031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e319c365-8692-480c-ba58-946afb06bdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:41.073598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27446c54-1375-49fc-b717-6f46cda339e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:41.074855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27446c54-1375-49fc-b717-6f46cda339e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.075148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27446c54-1375-49fc-b717-6f46cda339e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.075480 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27446c54-1375-49fc-b717-6f46cda339e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.076102 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27446c54-1375-49fc-b717-6f46cda339e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:41.078608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27446c54-1375-49fc-b717-6f46cda339e0 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:41.081624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1024/2037] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:41.090354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81393ab6-55f1-48e3-ab50-c9c362c7e7e4 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.090640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81393ab6-55f1-48e3-ab50-c9c362c7e7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.091465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81393ab6-55f1-48e3-ab50-c9c362c7e7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.104791 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1014/2038] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:25:40 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:25:41.108979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27446c54-1375-49fc-b717-6f46cda339e0 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:41.110845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-27446c54-1375-49fc-b717-6f46cda339e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:41.122499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27446c54-1375-49fc-b717-6f46cda339e0 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.125384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-81393ab6-55f1-48e3-ab50-c9c362c7e7e4 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:41.127060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-81393ab6-55f1-48e3-ab50-c9c362c7e7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:41.127360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea97bf1-da2c-4699-8482-53509266c016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:41.128071 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea97bf1-da2c-4699-8482-53509266c016 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.128272 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea97bf1-da2c-4699-8482-53509266c016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.128544 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea97bf1-da2c-4699-8482-53509266c016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.129243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea97bf1-da2c-4699-8482-53509266c016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:41.130927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea97bf1-da2c-4699-8482-53509266c016 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:41.137448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1025/2039] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:41.142051 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81393ab6-55f1-48e3-ab50-c9c362c7e7e4 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:41.144956 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1015/2040] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:41.148824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea97bf1-da2c-4699-8482-53509266c016 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:41.155471 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81393ab6-55f1-48e3-ab50-c9c362c7e7e4 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.161195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea97bf1-da2c-4699-8482-53509266c016 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.161581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd987c6-164a-44c9-b9e7-48f1ea7ab2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:41.162463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd987c6-164a-44c9-b9e7-48f1ea7ab2bc admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.162776 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd987c6-164a-44c9-b9e7-48f1ea7ab2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.163012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd987c6-164a-44c9-b9e7-48f1ea7ab2bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.163874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd987c6-164a-44c9-b9e7-48f1ea7ab2bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:25:41.164223 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd987c6-164a-44c9-b9e7-48f1ea7ab2bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:41.178097 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd987c6-164a-44c9-b9e7-48f1ea7ab2bc admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:41.180094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dcf90-bf0c-41d7-80c9-6455a1bc8790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:41.180780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dcf90-bf0c-41d7-80c9-6455a1bc8790 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.181093 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dcf90-bf0c-41d7-80c9-6455a1bc8790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.181422 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dcf90-bf0c-41d7-80c9-6455a1bc8790 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.182192 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68dcf90-bf0c-41d7-80c9-6455a1bc8790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:41.183911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68dcf90-bf0c-41d7-80c9-6455a1bc8790 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:41.190862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1026/2041] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:25:41.192483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71eb4d3-0efe-437a-9da7-9fe7fa5e03ba None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.192949 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71eb4d3-0efe-437a-9da7-9fe7fa5e03ba None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.193425 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71eb4d3-0efe-437a-9da7-9fe7fa5e03ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.195803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1016/2042] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:41.196590 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dcf90-bf0c-41d7-80c9-6455a1bc8790 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:41.208837 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dcf90-bf0c-41d7-80c9-6455a1bc8790 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.213816 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-f71eb4d3-0efe-437a-9da7-9fe7fa5e03ba None None] MFA Rules not processed for user `0898fda8be3743f093dbb216f21eb931`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:41.215054 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f71eb4d3-0efe-437a-9da7-9fe7fa5e03ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:41.216493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21544013-1a95-4f49-8687-4653a46ab1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:41.217128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21544013-1a95-4f49-8687-4653a46ab1ab admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.217464 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21544013-1a95-4f49-8687-4653a46ab1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.217722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21544013-1a95-4f49-8687-4653a46ab1ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.218411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21544013-1a95-4f49-8687-4653a46ab1ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:41.220406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21544013-1a95-4f49-8687-4653a46ab1ab admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:41.248529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1017/2043] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:25:41 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:41.249440 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21544013-1a95-4f49-8687-4653a46ab1ab admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:41.264216 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:41.278368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1027/2044] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:41.279241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71eb4d3-0efe-437a-9da7-9fe7fa5e03ba None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:41.296767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71eb4d3-0efe-437a-9da7-9fe7fa5e03ba None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.300784 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:41.320398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21544013-1a95-4f49-8687-4653a46ab1ab admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.325271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d7eba-588f-4698-bea2-648e0f8a5f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:41.326003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d7eba-588f-4698-bea2-648e0f8a5f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.326516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d7eba-588f-4698-bea2-648e0f8a5f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.326797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d7eba-588f-4698-bea2-648e0f8a5f29 admin admin] PATH_INFO: `/v3/projects/4c1b0e33810048ac954f34b039ba414f/users/b2560e7adf78474d967b055396b7fead/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.350477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d7eba-588f-4698-bea2-648e0f8a5f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c1b0e33810048ac954f34b039ba414f, user_id=b2560e7adf78474d967b055396b7fead, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:41.353160 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d7eba-588f-4698-bea2-648e0f8a5f29 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:41.361694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6c6dd-0e5f-4eec-8d0c-44b64135b590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:41.362696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6c6dd-0e5f-4eec-8d0c-44b64135b590 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.362981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6c6dd-0e5f-4eec-8d0c-44b64135b590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.363243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6c6dd-0e5f-4eec-8d0c-44b64135b590 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.363917 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6c6dd-0e5f-4eec-8d0c-44b64135b590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:41.366144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6c6dd-0e5f-4eec-8d0c-44b64135b590 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:41.377964 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1018/2045] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:41 2022] PUT /identity/v3/projects/4c1b0e33810048ac954f34b039ba414f/users/b2560e7adf78474d967b055396b7fead/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:41.386362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071587ce-5a5a-4372-9d7f-f9d543c456d7 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.386670 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071587ce-5a5a-4372-9d7f-f9d543c456d7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.387084 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071587ce-5a5a-4372-9d7f-f9d543c456d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.397852 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1028/2046] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:25:41 2022] POST /identity/v3/users => generated 297 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:41.402212 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6c6dd-0e5f-4eec-8d0c-44b64135b590 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:41.413892 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6c6dd-0e5f-4eec-8d0c-44b64135b590 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.418649 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-071587ce-5a5a-4372-9d7f-f9d543c456d7 None None] MFA Rules not processed for user `b2560e7adf78474d967b055396b7fead`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:41.420062 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-071587ce-5a5a-4372-9d7f-f9d543c456d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:41.486473 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e11073-8d92-497b-80f0-557f55ec9dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:41.487192 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e11073-8d92-497b-80f0-557f55ec9dfd admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.487508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e11073-8d92-497b-80f0-557f55ec9dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.487898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e11073-8d92-497b-80f0-557f55ec9dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.488674 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e11073-8d92-497b-80f0-557f55ec9dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:41.490527 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e11073-8d92-497b-80f0-557f55ec9dfd admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:41.503721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1019/2047] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:41.503927 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1029/2048] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:25:41.506741 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071587ce-5a5a-4372-9d7f-f9d543c456d7 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:41.506955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e11073-8d92-497b-80f0-557f55ec9dfd admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:41.517291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e11073-8d92-497b-80f0-557f55ec9dfd admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.518306 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071587ce-5a5a-4372-9d7f-f9d543c456d7 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.522184 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed6fee-1a6c-4259-8eed-fe2e383a9b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:41.522778 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed6fee-1a6c-4259-8eed-fe2e383a9b30 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.523057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed6fee-1a6c-4259-8eed-fe2e383a9b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.523344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed6fee-1a6c-4259-8eed-fe2e383a9b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.524099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed6fee-1a6c-4259-8eed-fe2e383a9b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:41.524859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5a46f-cc7b-4ead-8f82-d21f50c3be4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:41.525701 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5a46f-cc7b-4ead-8f82-d21f50c3be4a admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.525862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed6fee-1a6c-4259-8eed-fe2e383a9b30 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:41.526294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5a46f-cc7b-4ead-8f82-d21f50c3be4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.526714 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5a46f-cc7b-4ead-8f82-d21f50c3be4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.527530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd5a46f-cc7b-4ead-8f82-d21f50c3be4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:41.529607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd5a46f-cc7b-4ead-8f82-d21f50c3be4a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:41.537703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1030/2049] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:25:41.541881 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed6fee-1a6c-4259-8eed-fe2e383a9b30 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:41.553199 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed6fee-1a6c-4259-8eed-fe2e383a9b30 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.560227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee48dc3-3107-4881-b501-0e44192bcdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:41.560910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee48dc3-3107-4881-b501-0e44192bcdba admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.561192 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee48dc3-3107-4881-b501-0e44192bcdba admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.561493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee48dc3-3107-4881-b501-0e44192bcdba admin admin] PATH_INFO: `/v3/projects/245accee121c4e72a6080b42314d04c1/users/a67c7ffa8cbf4ee495928745961c8f68/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.564428 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1020/2050] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:25:41 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:41.568589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5a46f-cc7b-4ead-8f82-d21f50c3be4a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:41.582790 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee48dc3-3107-4881-b501-0e44192bcdba admin admin] RBAC: Authorizing `identity:create_grant(project_id=245accee121c4e72a6080b42314d04c1, user_id=a67c7ffa8cbf4ee495928745961c8f68, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:41.587456 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee48dc3-3107-4881-b501-0e44192bcdba admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:41.605420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1031/2051] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:41 2022] PUT /identity/v3/projects/245accee121c4e72a6080b42314d04c1/users/a67c7ffa8cbf4ee495928745961c8f68/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:41.617022 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b8ff4-0217-4650-9b4d-248f8fe0185f None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.617311 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b8ff4-0217-4650-9b4d-248f8fe0185f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.617589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b8ff4-0217-4650-9b4d-248f8fe0185f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.651458 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5a46f-cc7b-4ead-8f82-d21f50c3be4a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.652133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-6e8b8ff4-0217-4650-9b4d-248f8fe0185f None None] MFA Rules not processed for user `a67c7ffa8cbf4ee495928745961c8f68`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:41.653978 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6e8b8ff4-0217-4650-9b4d-248f8fe0185f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:41.698256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:41.700715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:41.725695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3eb3c-ead9-4fc2-a169-6d400af94c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:41.726525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3eb3c-ead9-4fc2-a169-6d400af94c90 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.726906 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3eb3c-ead9-4fc2-a169-6d400af94c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.727285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3eb3c-ead9-4fc2-a169-6d400af94c90 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.728036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3eb3c-ead9-4fc2-a169-6d400af94c90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:41.730245 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3eb3c-ead9-4fc2-a169-6d400af94c90 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:41.751640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1032/2052] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:41.752779 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b8ff4-0217-4650-9b4d-248f8fe0185f None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:41.761174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1021/2053] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:25:41 2022] POST /identity/v3/users => generated 295 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:25:41.762357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3eb3c-ead9-4fc2-a169-6d400af94c90 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:41.768549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b8ff4-0217-4650-9b4d-248f8fe0185f None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.773271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3eb3c-ead9-4fc2-a169-6d400af94c90 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.775023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed8a3f-0952-4bbe-9927-8fbf152f2a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:41.776360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed8a3f-0952-4bbe-9927-8fbf152f2a51 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.777589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed8a3f-0952-4bbe-9927-8fbf152f2a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.777987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed8a3f-0952-4bbe-9927-8fbf152f2a51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.778714 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed8a3f-0952-4bbe-9927-8fbf152f2a51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:41.781306 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e62c73-5758-43b8-9ee4-d81284e72564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:41.782107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e62c73-5758-43b8-9ee4-d81284e72564 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.782384 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e62c73-5758-43b8-9ee4-d81284e72564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.782807 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e62c73-5758-43b8-9ee4-d81284e72564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.783395 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed8a3f-0952-4bbe-9927-8fbf152f2a51 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:41.783576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e62c73-5758-43b8-9ee4-d81284e72564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:41.785242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e62c73-5758-43b8-9ee4-d81284e72564 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:41.802771 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1022/2054] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:25:41.803686 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e62c73-5758-43b8-9ee4-d81284e72564 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:41.815979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e62c73-5758-43b8-9ee4-d81284e72564 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.822670 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a05d9-db56-4bea-89e1-7734efd2bcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:41.823783 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a05d9-db56-4bea-89e1-7734efd2bcd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.823895 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a05d9-db56-4bea-89e1-7734efd2bcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.824123 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a05d9-db56-4bea-89e1-7734efd2bcd9 admin admin] PATH_INFO: `/v3/projects/f38d2728da624c8ca0cda2d698cbdfe9/users/89d7af67fca74cd5bab12c299ea69067/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.844754 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a05d9-db56-4bea-89e1-7734efd2bcd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f38d2728da624c8ca0cda2d698cbdfe9, user_id=89d7af67fca74cd5bab12c299ea69067, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:41.846261 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1033/2055] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:25:41 2022] POST /identity/v3/projects => generated 393 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:25:41.850259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed8a3f-0952-4bbe-9927-8fbf152f2a51 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:41.850671 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a05d9-db56-4bea-89e1-7734efd2bcd9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:41.880854 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1023/2056] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:41 2022] PUT /identity/v3/projects/f38d2728da624c8ca0cda2d698cbdfe9/users/89d7af67fca74cd5bab12c299ea69067/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:41.884934 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a05d9-db56-4bea-89e1-7734efd2bcd9 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:41.945411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed8a3f-0952-4bbe-9927-8fbf152f2a51 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.978129 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a05d9-db56-4bea-89e1-7734efd2bcd9 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:41.985046 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9200c1-0ae4-4289-852f-e466437069b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:41.985766 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9200c1-0ae4-4289-852f-e466437069b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:41.986349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9200c1-0ae4-4289-852f-e466437069b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:41.986719 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9200c1-0ae4-4289-852f-e466437069b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:41.987638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9200c1-0ae4-4289-852f-e466437069b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:41.989728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9200c1-0ae4-4289-852f-e466437069b5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:42.002922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1024/2057] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:41 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:42.007291 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9200c1-0ae4-4289-852f-e466437069b5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:42.019172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9200c1-0ae4-4289-852f-e466437069b5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:42.020675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b6dfb-e1f0-4845-b4e2-976f10d69f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:42.021467 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b6dfb-e1f0-4845-b4e2-976f10d69f18 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:42.021833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b6dfb-e1f0-4845-b4e2-976f10d69f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:42.022268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b6dfb-e1f0-4845-b4e2-976f10d69f18 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:42.023018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b6dfb-e1f0-4845-b4e2-976f10d69f18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:42.024719 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50c681-8c09-4872-9694-b8f9229cafeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:42.025097 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b6dfb-e1f0-4845-b4e2-976f10d69f18 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:42.025558 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50c681-8c09-4872-9694-b8f9229cafeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:42.025947 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50c681-8c09-4872-9694-b8f9229cafeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:42.026293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50c681-8c09-4872-9694-b8f9229cafeb admin admin] PATH_INFO: `/v3/projects/f38d2728da624c8ca0cda2d698cbdfe9/users/89d7af67fca74cd5bab12c299ea69067/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:42.028419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50c681-8c09-4872-9694-b8f9229cafeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f38d2728da624c8ca0cda2d698cbdfe9, user_id=89d7af67fca74cd5bab12c299ea69067, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:42.032293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50c681-8c09-4872-9694-b8f9229cafeb admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:42.049407 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1034/2058] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:25:41 2022] POST /identity/v3/users => generated 303 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:42.054463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b6dfb-e1f0-4845-b4e2-976f10d69f18 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:42.055102 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1025/2059] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:42 2022] PUT /identity/v3/projects/f38d2728da624c8ca0cda2d698cbdfe9/users/89d7af67fca74cd5bab12c299ea69067/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:42.065461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbb5ab-bec9-4b51-9d09-493ccc73679f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:42.065745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbb5ab-bec9-4b51-9d09-493ccc73679f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:42.066055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbb5ab-bec9-4b51-9d09-493ccc73679f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:42.070598 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b6dfb-e1f0-4845-b4e2-976f10d69f18 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:42.100051 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-10fbb5ab-bec9-4b51-9d09-493ccc73679f None None] MFA Rules not processed for user `89d7af67fca74cd5bab12c299ea69067`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:42.101480 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-10fbb5ab-bec9-4b51-9d09-493ccc73679f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:42.136226 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:42.168214 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab6809-bb65-490e-a402-947790516026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:42.168867 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab6809-bb65-490e-a402-947790516026 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:42.169149 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab6809-bb65-490e-a402-947790516026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:42.169422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab6809-bb65-490e-a402-947790516026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:42.170169 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab6809-bb65-490e-a402-947790516026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:42.171812 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab6809-bb65-490e-a402-947790516026 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:42.191739 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1035/2060] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:25:42.196472 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab6809-bb65-490e-a402-947790516026 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:42.213075 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab6809-bb65-490e-a402-947790516026 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:42.222224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ba9d8-c251-4170-acbc-f359156bdf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:42.223816 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ba9d8-c251-4170-acbc-f359156bdf66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:42.223816 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ba9d8-c251-4170-acbc-f359156bdf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:42.223816 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ba9d8-c251-4170-acbc-f359156bdf66 admin admin] PATH_INFO: `/v3/projects/a7035a32bcd241f7b46587e1967aaa8c/users/b752d1af8efd408a8b52eb872971d2a6/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:42.231256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1026/2061] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:42.243922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbb5ab-bec9-4b51-9d09-493ccc73679f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:42.246210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-10fbb5ab-bec9-4b51-9d09-493ccc73679f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:42.254627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:42.259928 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ba9d8-c251-4170-acbc-f359156bdf66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7035a32bcd241f7b46587e1967aaa8c, user_id=b752d1af8efd408a8b52eb872971d2a6, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:42.265274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ba9d8-c251-4170-acbc-f359156bdf66 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:42.280753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1036/2062] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:42 2022] PUT /identity/v3/projects/a7035a32bcd241f7b46587e1967aaa8c/users/b752d1af8efd408a8b52eb872971d2a6/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:42.284545 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ba9d8-c251-4170-acbc-f359156bdf66 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:42.338816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbb5ab-bec9-4b51-9d09-493ccc73679f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:42.368711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ba9d8-c251-4170-acbc-f359156bdf66 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:42.377515 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77845daa-2537-4a36-a24d-5043917aa1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:42.378401 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77845daa-2537-4a36-a24d-5043917aa1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:42.378757 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77845daa-2537-4a36-a24d-5043917aa1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:42.379110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77845daa-2537-4a36-a24d-5043917aa1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:42.380101 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77845daa-2537-4a36-a24d-5043917aa1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:42.386514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77845daa-2537-4a36-a24d-5043917aa1a7 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:42.401698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1037/2063] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:25:42.406809 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77845daa-2537-4a36-a24d-5043917aa1a7 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:42.422482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77845daa-2537-4a36-a24d-5043917aa1a7 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:42.431133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69ef8a-4432-4121-9da5-2f48bc1173ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:42.431833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69ef8a-4432-4121-9da5-2f48bc1173ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:42.432120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69ef8a-4432-4121-9da5-2f48bc1173ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:42.432389 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69ef8a-4432-4121-9da5-2f48bc1173ed admin admin] PATH_INFO: `/v3/projects/a7035a32bcd241f7b46587e1967aaa8c/users/b752d1af8efd408a8b52eb872971d2a6/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:42.437099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69ef8a-4432-4121-9da5-2f48bc1173ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7035a32bcd241f7b46587e1967aaa8c, user_id=b752d1af8efd408a8b52eb872971d2a6, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:42.440657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69ef8a-4432-4121-9da5-2f48bc1173ed admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:42.448435 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a156b9d-1759-493b-9ec0-cbe7e091bb55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:42.449213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a156b9d-1759-493b-9ec0-cbe7e091bb55 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:42.449643 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a156b9d-1759-493b-9ec0-cbe7e091bb55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:42.450035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a156b9d-1759-493b-9ec0-cbe7e091bb55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:42.467483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1038/2064] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:42 2022] PUT /identity/v3/projects/a7035a32bcd241f7b46587e1967aaa8c/users/b752d1af8efd408a8b52eb872971d2a6/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:42.480355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434d851-7da9-43d7-b3ae-ccdedc54ac26 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:42.480650 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434d851-7da9-43d7-b3ae-ccdedc54ac26 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:42.480931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434d851-7da9-43d7-b3ae-ccdedc54ac26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:42.527013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-f434d851-7da9-43d7-b3ae-ccdedc54ac26 None None] MFA Rules not processed for user `b752d1af8efd408a8b52eb872971d2a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:42.529226 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f434d851-7da9-43d7-b3ae-ccdedc54ac26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:42.560523 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a156b9d-1759-493b-9ec0-cbe7e091bb55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:42.562297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a156b9d-1759-493b-9ec0-cbe7e091bb55 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:42.574731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1027/2065] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:25:42.624334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1039/2066] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:42.629082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a156b9d-1759-493b-9ec0-cbe7e091bb55 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:42.638665 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a156b9d-1759-493b-9ec0-cbe7e091bb55 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:42.645723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434d851-7da9-43d7-b3ae-ccdedc54ac26 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:42.647384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f434d851-7da9-43d7-b3ae-ccdedc54ac26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:42.688318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:42.693928 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537dc5b-c040-41da-aa1c-c3882e4b2340 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:42.694599 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537dc5b-c040-41da-aa1c-c3882e4b2340 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:42.694941 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537dc5b-c040-41da-aa1c-c3882e4b2340 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:42.695278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537dc5b-c040-41da-aa1c-c3882e4b2340 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:42.700114 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1537dc5b-c040-41da-aa1c-c3882e4b2340 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:42.726501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434d851-7da9-43d7-b3ae-ccdedc54ac26 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:42.731853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cb9e0-7b29-4677-9de6-a20e78b699da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:42.732752 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cb9e0-7b29-4677-9de6-a20e78b699da service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:42.733667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:42.734010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cb9e0-7b29-4677-9de6-a20e78b699da service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:42.734359 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cb9e0-7b29-4677-9de6-a20e78b699da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:42.750316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cb9e0-7b29-4677-9de6-a20e78b699da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:42.752217 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709cb9e0-7b29-4677-9de6-a20e78b699da service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:42.763881 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1040/2067] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:25:42.772962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537dc5b-c040-41da-aa1c-c3882e4b2340 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:42.774784 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537dc5b-c040-41da-aa1c-c3882e4b2340 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:42.785332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1028/2068] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:42.835081 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cb9e0-7b29-4677-9de6-a20e78b699da service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:42.849427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709cb9e0-7b29-4677-9de6-a20e78b699da service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:42.849900 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c25874-a59b-4972-8f3a-da94f309d87c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:42.850643 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c25874-a59b-4972-8f3a-da94f309d87c service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:42.851009 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c25874-a59b-4972-8f3a-da94f309d87c service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:42.851268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c25874-a59b-4972-8f3a-da94f309d87c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:42.955392 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c25874-a59b-4972-8f3a-da94f309d87c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:42.957095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c25874-a59b-4972-8f3a-da94f309d87c service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:42.971258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1041/2069] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:25:43.575224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50635ae-93e4-49f2-95c3-81cdfdaf3890 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:43.575723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50635ae-93e4-49f2-95c3-81cdfdaf3890 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:43.575816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50635ae-93e4-49f2-95c3-81cdfdaf3890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:43.595723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b50635ae-93e4-49f2-95c3-81cdfdaf3890 None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:43.596935 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b50635ae-93e4-49f2-95c3-81cdfdaf3890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:43.657523 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1029/2070] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 15:25:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:43.706917 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c25874-a59b-4972-8f3a-da94f309d87c service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:43.708365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-68c25874-a59b-4972-8f3a-da94f309d87c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:43.783042 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c25874-a59b-4972-8f3a-da94f309d87c service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:43.789701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef677eca-7a98-447c-8713-96bb24694026 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:43.790392 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef677eca-7a98-447c-8713-96bb24694026 service swift] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:43.790698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef677eca-7a98-447c-8713-96bb24694026 service swift] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:43.790985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef677eca-7a98-447c-8713-96bb24694026 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:43.799489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef677eca-7a98-447c-8713-96bb24694026 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:43.801104 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef677eca-7a98-447c-8713-96bb24694026 service swift] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:43.809602 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1042/2071] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:25:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:25:44.422006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8406f-aaf5-4c6a-b3cb-83d76b01fe08 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:44.422006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8406f-aaf5-4c6a-b3cb-83d76b01fe08 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:44.422006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8406f-aaf5-4c6a-b3cb-83d76b01fe08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:44.450944 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f9f8406f-aaf5-4c6a-b3cb-83d76b01fe08 None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:44.453883 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f9f8406f-aaf5-4c6a-b3cb-83d76b01fe08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:44.463855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1030/2072] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 15:25:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:44.488738 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef677eca-7a98-447c-8713-96bb24694026 service swift] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:44.508904 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef677eca-7a98-447c-8713-96bb24694026 service swift] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:44.516746 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fde352-b25f-4230-a510-5c981b41cae0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:44.517917 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fde352-b25f-4230-a510-5c981b41cae0 service swift] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:44.518306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fde352-b25f-4230-a510-5c981b41cae0 service swift] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:44.518676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fde352-b25f-4230-a510-5c981b41cae0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:44.532516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fde352-b25f-4230-a510-5c981b41cae0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:44.534633 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fde352-b25f-4230-a510-5c981b41cae0 service swift] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:44.549823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1043/2073] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:25:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:25:45.023675 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851f7f6-1b6c-4b60-ac53-2b578f32c015 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:45.024123 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851f7f6-1b6c-4b60-ac53-2b578f32c015 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:45.024215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851f7f6-1b6c-4b60-ac53-2b578f32c015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:45.062912 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-0851f7f6-1b6c-4b60-ac53-2b578f32c015 None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:45.064151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0851f7f6-1b6c-4b60-ac53-2b578f32c015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:45.086677 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1031/2074] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 15:25:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:45.105106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fde352-b25f-4230-a510-5c981b41cae0 service swift] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:45.129074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fde352-b25f-4230-a510-5c981b41cae0 service swift] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:45.137681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6e51d-693e-43fe-bcb8-816a1bf3582f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:45.138627 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6e51d-693e-43fe-bcb8-816a1bf3582f service swift] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:45.139054 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6e51d-693e-43fe-bcb8-816a1bf3582f service swift] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:45.139492 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6e51d-693e-43fe-bcb8-816a1bf3582f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:45.153182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6e51d-693e-43fe-bcb8-816a1bf3582f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:45.156528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6e51d-693e-43fe-bcb8-816a1bf3582f service swift] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:45.172174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1044/2075] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:25:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:25:45.655878 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d61e62-9294-45e8-9024-d7bb515af148 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:45.656325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d61e62-9294-45e8-9024-d7bb515af148 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:45.656702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d61e62-9294-45e8-9024-d7bb515af148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:45.676232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-85d61e62-9294-45e8-9024-d7bb515af148 None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:45.677524 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-85d61e62-9294-45e8-9024-d7bb515af148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:45.689117 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1032/2076] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 15:25:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:45.707318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6e51d-693e-43fe-bcb8-816a1bf3582f service swift] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:45.738145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6e51d-693e-43fe-bcb8-816a1bf3582f service swift] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:45.744460 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fd575-9af9-4ea7-93f4-c8c9fa3a7b4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:45.746814 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fd575-9af9-4ea7-93f4-c8c9fa3a7b4f service swift] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:45.747287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fd575-9af9-4ea7-93f4-c8c9fa3a7b4f service swift] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:45.747750 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fd575-9af9-4ea7-93f4-c8c9fa3a7b4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:45.760334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710fd575-9af9-4ea7-93f4-c8c9fa3a7b4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:45.764608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710fd575-9af9-4ea7-93f4-c8c9fa3a7b4f service swift] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:45.777833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1045/2077] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:25:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:25:46.325128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d61e62-9294-45e8-9024-d7bb515af148 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:46.429452 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d61e62-9294-45e8-9024-d7bb515af148 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:46.434451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa4baea-d5e4-48a7-b607-e187e1644472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:46.435152 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4baea-d5e4-48a7-b607-e187e1644472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:46.435417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4baea-d5e4-48a7-b607-e187e1644472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:46.435845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa4baea-d5e4-48a7-b607-e187e1644472 admin admin] PATH_INFO: `/v3/users/f9c2d238ddb64745addce164c10f8a40` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:46.437898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa4baea-d5e4-48a7-b607-e187e1644472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9c2d238ddb64745addce164c10f8a40)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:46.440057 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa4baea-d5e4-48a7-b607-e187e1644472 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:46.495836 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-efa4baea-d5e4-48a7-b607-e187e1644472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9c2d238ddb64745addce164c10f8a40'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:46.496876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-efa4baea-d5e4-48a7-b607-e187e1644472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9c2d238ddb64745addce164c10f8a40'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:46.501578 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:46.511008 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-efa4baea-d5e4-48a7-b607-e187e1644472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9c2d238ddb64745addce164c10f8a40'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:46.519592 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-efa4baea-d5e4-48a7-b607-e187e1644472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9c2d238ddb64745addce164c10f8a40'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:46.530324 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-efa4baea-d5e4-48a7-b607-e187e1644472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9c2d238ddb64745addce164c10f8a40'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:46.536355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1033/2078] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:25:46 2022] DELETE /identity/v3/users/f9c2d238ddb64745addce164c10f8a40 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:46.540871 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b738edc-13e6-4e26-b3fb-bbca1642c198 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:46.541366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b738edc-13e6-4e26-b3fb-bbca1642c198 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:46.541457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b738edc-13e6-4e26-b3fb-bbca1642c198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:46.562369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-1b738edc-13e6-4e26-b3fb-bbca1642c198 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:46.562369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1b738edc-13e6-4e26-b3fb-bbca1642c198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:46.575733 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:46.630291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1046/2079] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:46.645582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa4baea-d5e4-48a7-b607-e187e1644472 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:46.648232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-efa4baea-d5e4-48a7-b607-e187e1644472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:46.729483 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa4baea-d5e4-48a7-b607-e187e1644472 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:46.734250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff2fc3-fc6b-417f-8586-b0df2b4775ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:46.735577 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff2fc3-fc6b-417f-8586-b0df2b4775ff service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:46.736108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff2fc3-fc6b-417f-8586-b0df2b4775ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:46.736579 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff2fc3-fc6b-417f-8586-b0df2b4775ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:46.753763 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff2fc3-fc6b-417f-8586-b0df2b4775ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:46.755609 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff2fc3-fc6b-417f-8586-b0df2b4775ff service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:46.779313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1034/2080] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:47.057966 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b738edc-13e6-4e26-b3fb-bbca1642c198 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:47.060833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1b738edc-13e6-4e26-b3fb-bbca1642c198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:47.074018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b738edc-13e6-4e26-b3fb-bbca1642c198 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:47.078923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54157ccb-53b1-4b12-b27f-6fa338195974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:47.079671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54157ccb-53b1-4b12-b27f-6fa338195974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:47.080015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54157ccb-53b1-4b12-b27f-6fa338195974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:47.080356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54157ccb-53b1-4b12-b27f-6fa338195974 admin admin] PATH_INFO: `/v3/users/a67c7ffa8cbf4ee495928745961c8f68` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:47.081718 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54157ccb-53b1-4b12-b27f-6fa338195974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a67c7ffa8cbf4ee495928745961c8f68)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:47.084430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54157ccb-53b1-4b12-b27f-6fa338195974 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:47.142604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-54157ccb-53b1-4b12-b27f-6fa338195974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a67c7ffa8cbf4ee495928745961c8f68'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.152758 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-54157ccb-53b1-4b12-b27f-6fa338195974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a67c7ffa8cbf4ee495928745961c8f68'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.153812 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-54157ccb-53b1-4b12-b27f-6fa338195974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a67c7ffa8cbf4ee495928745961c8f68'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.166567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-54157ccb-53b1-4b12-b27f-6fa338195974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a67c7ffa8cbf4ee495928745961c8f68'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.196886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-54157ccb-53b1-4b12-b27f-6fa338195974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a67c7ffa8cbf4ee495928745961c8f68'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.212154 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1047/2081] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:25:47 2022] DELETE /identity/v3/users/a67c7ffa8cbf4ee495928745961c8f68 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:47.216690 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8707832-6483-4f09-a43f-ec20679c3956 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:47.217318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8707832-6483-4f09-a43f-ec20679c3956 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:47.217432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8707832-6483-4f09-a43f-ec20679c3956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:47.245773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d8707832-6483-4f09-a43f-ec20679c3956 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:47.247788 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d8707832-6483-4f09-a43f-ec20679c3956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:47.325975 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1035/2082] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:47.335376 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54157ccb-53b1-4b12-b27f-6fa338195974 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:47.337411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-54157ccb-53b1-4b12-b27f-6fa338195974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:47.355415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8707832-6483-4f09-a43f-ec20679c3956 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:47.356935 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d8707832-6483-4f09-a43f-ec20679c3956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:47.370429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8707832-6483-4f09-a43f-ec20679c3956 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:47.380738 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa54c4a-b667-433b-a60b-23eed298d161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:47.381444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa54c4a-b667-433b-a60b-23eed298d161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:47.381815 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa54c4a-b667-433b-a60b-23eed298d161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:47.382053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa54c4a-b667-433b-a60b-23eed298d161 admin admin] PATH_INFO: `/v3/projects/3efda1dbd08544eda0f5cea5e3dbe356` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:47.384541 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa54c4a-b667-433b-a60b-23eed298d161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3efda1dbd08544eda0f5cea5e3dbe356)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:47.386897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa54c4a-b667-433b-a60b-23eed298d161 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:47.442224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54157ccb-53b1-4b12-b27f-6fa338195974 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:47.452691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-afa54c4a-b667-433b-a60b-23eed298d161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3efda1dbd08544eda0f5cea5e3dbe356', 'request_id': 'req-afa54c4a-b667-433b-a60b-23eed298d161'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.458583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-afa54c4a-b667-433b-a60b-23eed298d161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3efda1dbd08544eda0f5cea5e3dbe356', 'request_id': 'req-afa54c4a-b667-433b-a60b-23eed298d161'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.468578 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-afa54c4a-b667-433b-a60b-23eed298d161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3efda1dbd08544eda0f5cea5e3dbe356', 'request_id': 'req-afa54c4a-b667-433b-a60b-23eed298d161'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.472072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:47.487522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-afa54c4a-b667-433b-a60b-23eed298d161 admin admin] The token cache is being invalidate because project 3efda1dbd08544eda0f5cea5e3dbe356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:25:47.487808 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-afa54c4a-b667-433b-a60b-23eed298d161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.488560 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-afa54c4a-b667-433b-a60b-23eed298d161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.491052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1036/2083] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:25:47 2022] DELETE /identity/v3/projects/3efda1dbd08544eda0f5cea5e3dbe356 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:47.495020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa54c4a-b667-433b-a60b-23eed298d161 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:47.496511 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-afa54c4a-b667-433b-a60b-23eed298d161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:47.509520 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:47.516885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb7156-93d2-499f-bd69-4e1767083045 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:47.517960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb7156-93d2-499f-bd69-4e1767083045 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:47.518621 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb7156-93d2-499f-bd69-4e1767083045 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:47.518866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb7156-93d2-499f-bd69-4e1767083045 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:47.521558 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-eabb7156-93d2-499f-bd69-4e1767083045 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:47.571013 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa54c4a-b667-433b-a60b-23eed298d161 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:47.578052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae480799-aba4-4723-adf3-3c8d7c59cd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:47.578972 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae480799-aba4-4723-adf3-3c8d7c59cd46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:47.579390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae480799-aba4-4723-adf3-3c8d7c59cd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:47.579826 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae480799-aba4-4723-adf3-3c8d7c59cd46 admin admin] PATH_INFO: `/v3/users/b2560e7adf78474d967b055396b7fead` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:47.581609 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae480799-aba4-4723-adf3-3c8d7c59cd46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2560e7adf78474d967b055396b7fead)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:47.584837 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae480799-aba4-4723-adf3-3c8d7c59cd46 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:47.608072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb7156-93d2-499f-bd69-4e1767083045 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:47.609528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb7156-93d2-499f-bd69-4e1767083045 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:47.622741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1048/2084] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:47.636824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ae480799-aba4-4723-adf3-3c8d7c59cd46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2560e7adf78474d967b055396b7fead'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.637556 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ae480799-aba4-4723-adf3-3c8d7c59cd46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2560e7adf78474d967b055396b7fead'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.651993 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ae480799-aba4-4723-adf3-3c8d7c59cd46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2560e7adf78474d967b055396b7fead'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.662848 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ae480799-aba4-4723-adf3-3c8d7c59cd46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2560e7adf78474d967b055396b7fead'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.673488 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ae480799-aba4-4723-adf3-3c8d7c59cd46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2560e7adf78474d967b055396b7fead'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:47.684912 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1037/2085] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:25:47 2022] DELETE /identity/v3/users/b2560e7adf78474d967b055396b7fead => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:48.096997 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb7156-93d2-499f-bd69-4e1767083045 service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:48.098701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-eabb7156-93d2-499f-bd69-4e1767083045 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:48.117033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb7156-93d2-499f-bd69-4e1767083045 service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:48.172539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae480799-aba4-4723-adf3-3c8d7c59cd46 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:48.174416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ae480799-aba4-4723-adf3-3c8d7c59cd46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:48.237364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62f937-a3f8-4983-807b-d12c6f325418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:48.238504 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62f937-a3f8-4983-807b-d12c6f325418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.239097 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62f937-a3f8-4983-807b-d12c6f325418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.239476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa62f937-a3f8-4983-807b-d12c6f325418 admin admin] PATH_INFO: `/v3/users/0b8be63e9d494662b9ca0cabf01e86ad` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.241637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62f937-a3f8-4983-807b-d12c6f325418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b8be63e9d494662b9ca0cabf01e86ad)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:48.245418 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa62f937-a3f8-4983-807b-d12c6f325418 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:48.271119 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae480799-aba4-4723-adf3-3c8d7c59cd46 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:48.279579 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c88f44-09e3-4fe1-aa41-430577fa1eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:48.280898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c88f44-09e3-4fe1-aa41-430577fa1eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.281471 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c88f44-09e3-4fe1-aa41-430577fa1eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.281967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c88f44-09e3-4fe1-aa41-430577fa1eef admin admin] PATH_INFO: `/v3/projects/4c1b0e33810048ac954f34b039ba414f` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.284644 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c88f44-09e3-4fe1-aa41-430577fa1eef admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c1b0e33810048ac954f34b039ba414f)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:48.288271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c88f44-09e3-4fe1-aa41-430577fa1eef admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:48.310724 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-aa62f937-a3f8-4983-807b-d12c6f325418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b8be63e9d494662b9ca0cabf01e86ad'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.323509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-aa62f937-a3f8-4983-807b-d12c6f325418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b8be63e9d494662b9ca0cabf01e86ad'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.323509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-aa62f937-a3f8-4983-807b-d12c6f325418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b8be63e9d494662b9ca0cabf01e86ad'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.324528 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:48.342989 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-aa62f937-a3f8-4983-807b-d12c6f325418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b8be63e9d494662b9ca0cabf01e86ad'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.363162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-aa62f937-a3f8-4983-807b-d12c6f325418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b8be63e9d494662b9ca0cabf01e86ad'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.369852 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1049/2086] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:25:48 2022] DELETE /identity/v3/users/0b8be63e9d494662b9ca0cabf01e86ad => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:48.370558 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62f937-a3f8-4983-807b-d12c6f325418 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:48.372109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-aa62f937-a3f8-4983-807b-d12c6f325418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:48.386790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c8c88f44-09e3-4fe1-aa41-430577fa1eef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c1b0e33810048ac954f34b039ba414f', 'request_id': 'req-c8c88f44-09e3-4fe1-aa41-430577fa1eef'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.405099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:48.406318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c8c88f44-09e3-4fe1-aa41-430577fa1eef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c1b0e33810048ac954f34b039ba414f', 'request_id': 'req-c8c88f44-09e3-4fe1-aa41-430577fa1eef'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.423850 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c8c88f44-09e3-4fe1-aa41-430577fa1eef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c1b0e33810048ac954f34b039ba414f', 'request_id': 'req-c8c88f44-09e3-4fe1-aa41-430577fa1eef'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.441548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c8c88f44-09e3-4fe1-aa41-430577fa1eef admin admin] The token cache is being invalidate because project 4c1b0e33810048ac954f34b039ba414f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:25:48.442217 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c8c88f44-09e3-4fe1-aa41-430577fa1eef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.443116 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c8c88f44-09e3-4fe1-aa41-430577fa1eef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.445129 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1038/2087] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:25:48 2022] DELETE /identity/v3/projects/4c1b0e33810048ac954f34b039ba414f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:48.446687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a723bdc-8c03-455d-9215-20a44045bebb None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.447284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a723bdc-8c03-455d-9215-20a44045bebb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.448003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a723bdc-8c03-455d-9215-20a44045bebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.470622 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2a723bdc-8c03-455d-9215-20a44045bebb None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:48.472579 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2a723bdc-8c03-455d-9215-20a44045bebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:48.501458 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa62f937-a3f8-4983-807b-d12c6f325418 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:48.507571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86b748-a283-4998-a0c9-246f8d6a8747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:48.508471 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86b748-a283-4998-a0c9-246f8d6a8747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.508851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86b748-a283-4998-a0c9-246f8d6a8747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.509128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86b748-a283-4998-a0c9-246f8d6a8747 admin admin] PATH_INFO: `/v3/projects/245accee121c4e72a6080b42314d04c1` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.510448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86b748-a283-4998-a0c9-246f8d6a8747 admin admin] RBAC: Authorizing `identity:delete_project(project_id=245accee121c4e72a6080b42314d04c1)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:48.512459 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86b748-a283-4998-a0c9-246f8d6a8747 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:48.553879 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1039/2088] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:48.555154 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a723bdc-8c03-455d-9215-20a44045bebb None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:48.556479 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2a723bdc-8c03-455d-9215-20a44045bebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:48.568249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3d86b748-a283-4998-a0c9-246f8d6a8747 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '245accee121c4e72a6080b42314d04c1', 'request_id': 'req-3d86b748-a283-4998-a0c9-246f8d6a8747'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.570415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a723bdc-8c03-455d-9215-20a44045bebb None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:48.576859 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3d86b748-a283-4998-a0c9-246f8d6a8747 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '245accee121c4e72a6080b42314d04c1', 'request_id': 'req-3d86b748-a283-4998-a0c9-246f8d6a8747'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.584960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3d86b748-a283-4998-a0c9-246f8d6a8747 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '245accee121c4e72a6080b42314d04c1', 'request_id': 'req-3d86b748-a283-4998-a0c9-246f8d6a8747'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.598819 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3d86b748-a283-4998-a0c9-246f8d6a8747 admin admin] The token cache is being invalidate because project 245accee121c4e72a6080b42314d04c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:25:48.599184 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3d86b748-a283-4998-a0c9-246f8d6a8747 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.600399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3d86b748-a283-4998-a0c9-246f8d6a8747 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.605266 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1050/2089] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:25:48 2022] DELETE /identity/v3/projects/245accee121c4e72a6080b42314d04c1 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:48.606511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86b748-a283-4998-a0c9-246f8d6a8747 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:48.608135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-3d86b748-a283-4998-a0c9-246f8d6a8747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:48.656607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9947016-34ab-4d20-b947-c8cf2d7d18d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:48.657979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9947016-34ab-4d20-b947-c8cf2d7d18d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.658547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9947016-34ab-4d20-b947-c8cf2d7d18d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.659060 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9947016-34ab-4d20-b947-c8cf2d7d18d9 admin admin] PATH_INFO: `/v3/users/89d7af67fca74cd5bab12c299ea69067` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.660857 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9947016-34ab-4d20-b947-c8cf2d7d18d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89d7af67fca74cd5bab12c299ea69067)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:48.664905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9947016-34ab-4d20-b947-c8cf2d7d18d9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:48.692671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86b748-a283-4998-a0c9-246f8d6a8747 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:48.698065 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276807f-7367-4a1b-8427-fbb038415288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:48.699908 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276807f-7367-4a1b-8427-fbb038415288 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.700289 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276807f-7367-4a1b-8427-fbb038415288 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.700923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276807f-7367-4a1b-8427-fbb038415288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.703490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-7276807f-7367-4a1b-8427-fbb038415288 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:48.723439 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276807f-7367-4a1b-8427-fbb038415288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:48.728544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276807f-7367-4a1b-8427-fbb038415288 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:48.740111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e9947016-34ab-4d20-b947-c8cf2d7d18d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89d7af67fca74cd5bab12c299ea69067'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.740825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e9947016-34ab-4d20-b947-c8cf2d7d18d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89d7af67fca74cd5bab12c299ea69067'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.742162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:48.750455 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1051/2090] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:48.759465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276807f-7367-4a1b-8427-fbb038415288 service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:48.770534 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-7276807f-7367-4a1b-8427-fbb038415288 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:48.776585 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e9947016-34ab-4d20-b947-c8cf2d7d18d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89d7af67fca74cd5bab12c299ea69067'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.795032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e9947016-34ab-4d20-b947-c8cf2d7d18d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89d7af67fca74cd5bab12c299ea69067'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.802598 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276807f-7367-4a1b-8427-fbb038415288 service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:48.819257 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e8f67-5bfe-4987-9649-9e0ef831b48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:48.820544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e8f67-5bfe-4987-9649-9e0ef831b48f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:48.821340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e9947016-34ab-4d20-b947-c8cf2d7d18d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89d7af67fca74cd5bab12c299ea69067'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.822417 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e8f67-5bfe-4987-9649-9e0ef831b48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:48.822925 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e8f67-5bfe-4987-9649-9e0ef831b48f admin admin] PATH_INFO: `/v3/users/b752d1af8efd408a8b52eb872971d2a6` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:48.824523 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e8f67-5bfe-4987-9649-9e0ef831b48f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b752d1af8efd408a8b52eb872971d2a6)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:48.826826 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e8f67-5bfe-4987-9649-9e0ef831b48f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:48.834372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1040/2091] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:25:48 2022] DELETE /identity/v3/users/89d7af67fca74cd5bab12c299ea69067 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:48.915450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:48.920438 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-845e8f67-5bfe-4987-9649-9e0ef831b48f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b752d1af8efd408a8b52eb872971d2a6'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.934802 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-845e8f67-5bfe-4987-9649-9e0ef831b48f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b752d1af8efd408a8b52eb872971d2a6'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.936364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-845e8f67-5bfe-4987-9649-9e0ef831b48f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b752d1af8efd408a8b52eb872971d2a6'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.949433 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-845e8f67-5bfe-4987-9649-9e0ef831b48f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b752d1af8efd408a8b52eb872971d2a6'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.978261 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-845e8f67-5bfe-4987-9649-9e0ef831b48f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b752d1af8efd408a8b52eb872971d2a6'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:48.993655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1052/2092] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:25:48 2022] DELETE /identity/v3/users/b752d1af8efd408a8b52eb872971d2a6 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:49.545227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9947016-34ab-4d20-b947-c8cf2d7d18d9 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:49.547761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e9947016-34ab-4d20-b947-c8cf2d7d18d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:49.626766 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9947016-34ab-4d20-b947-c8cf2d7d18d9 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:49.698318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40e24e-55b1-4c59-85ba-6d86c32e659e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:49.699004 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40e24e-55b1-4c59-85ba-6d86c32e659e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:49.699296 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40e24e-55b1-4c59-85ba-6d86c32e659e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:49.699569 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40e24e-55b1-4c59-85ba-6d86c32e659e admin admin] PATH_INFO: `/v3/projects/4ad608aec8b94f28aba65bfabd8e51e3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:49.700756 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40e24e-55b1-4c59-85ba-6d86c32e659e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ad608aec8b94f28aba65bfabd8e51e3)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:49.702736 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40e24e-55b1-4c59-85ba-6d86c32e659e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:49.758775 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9d40e24e-55b1-4c59-85ba-6d86c32e659e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ad608aec8b94f28aba65bfabd8e51e3', 'request_id': 'req-9d40e24e-55b1-4c59-85ba-6d86c32e659e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:49.767122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9d40e24e-55b1-4c59-85ba-6d86c32e659e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ad608aec8b94f28aba65bfabd8e51e3', 'request_id': 'req-9d40e24e-55b1-4c59-85ba-6d86c32e659e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:49.778598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9d40e24e-55b1-4c59-85ba-6d86c32e659e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ad608aec8b94f28aba65bfabd8e51e3', 'request_id': 'req-9d40e24e-55b1-4c59-85ba-6d86c32e659e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:49.800390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9d40e24e-55b1-4c59-85ba-6d86c32e659e admin admin] The token cache is being invalidate because project 4ad608aec8b94f28aba65bfabd8e51e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:25:49.801515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9d40e24e-55b1-4c59-85ba-6d86c32e659e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:49.802650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9d40e24e-55b1-4c59-85ba-6d86c32e659e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:49.817932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1041/2093] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:25:49 2022] DELETE /identity/v3/projects/4ad608aec8b94f28aba65bfabd8e51e3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:49.838325 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e8f67-5bfe-4987-9649-9e0ef831b48f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:49.841323 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-845e8f67-5bfe-4987-9649-9e0ef831b48f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:49.885345 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e8f67-5bfe-4987-9649-9e0ef831b48f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:49.974936 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40e24e-55b1-4c59-85ba-6d86c32e659e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:49.976672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9d40e24e-55b1-4c59-85ba-6d86c32e659e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:49.980584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a97ada-0e09-4756-9108-7449ce926eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:49.981850 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a97ada-0e09-4756-9108-7449ce926eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:49.982441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a97ada-0e09-4756-9108-7449ce926eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:49.983203 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a97ada-0e09-4756-9108-7449ce926eca admin admin] PATH_INFO: `/v3/projects/f38d2728da624c8ca0cda2d698cbdfe9` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:49.984716 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a97ada-0e09-4756-9108-7449ce926eca admin admin] RBAC: Authorizing `identity:delete_project(project_id=f38d2728da624c8ca0cda2d698cbdfe9)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:49.987423 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a97ada-0e09-4756-9108-7449ce926eca admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:49.995908 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:50.005152 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40e24e-55b1-4c59-85ba-6d86c32e659e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.014917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22efa7-213c-4608-bf64-390132f06598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.016509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22efa7-213c-4608-bf64-390132f06598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.016958 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22efa7-213c-4608-bf64-390132f06598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.017370 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22efa7-213c-4608-bf64-390132f06598 admin admin] PATH_INFO: `/v3/projects/a7035a32bcd241f7b46587e1967aaa8c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.019110 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22efa7-213c-4608-bf64-390132f06598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7035a32bcd241f7b46587e1967aaa8c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.022537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22efa7-213c-4608-bf64-390132f06598 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.054253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d8a97ada-0e09-4756-9108-7449ce926eca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f38d2728da624c8ca0cda2d698cbdfe9', 'request_id': 'req-d8a97ada-0e09-4756-9108-7449ce926eca'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:50.061570 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d8a97ada-0e09-4756-9108-7449ce926eca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f38d2728da624c8ca0cda2d698cbdfe9', 'request_id': 'req-d8a97ada-0e09-4756-9108-7449ce926eca'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:50.068806 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d8a97ada-0e09-4756-9108-7449ce926eca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f38d2728da624c8ca0cda2d698cbdfe9', 'request_id': 'req-d8a97ada-0e09-4756-9108-7449ce926eca'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:50.074084 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7a22efa7-213c-4608-bf64-390132f06598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7035a32bcd241f7b46587e1967aaa8c', 'request_id': 'req-7a22efa7-213c-4608-bf64-390132f06598'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:50.079528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d8a97ada-0e09-4756-9108-7449ce926eca admin admin] The token cache is being invalidate because project f38d2728da624c8ca0cda2d698cbdfe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:25:50.079807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d8a97ada-0e09-4756-9108-7449ce926eca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:50.080437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d8a97ada-0e09-4756-9108-7449ce926eca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:50.080688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7a22efa7-213c-4608-bf64-390132f06598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7035a32bcd241f7b46587e1967aaa8c', 'request_id': 'req-7a22efa7-213c-4608-bf64-390132f06598'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:50.081975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1053/2094] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:25:49 2022] DELETE /identity/v3/projects/f38d2728da624c8ca0cda2d698cbdfe9 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:50.088548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448a665-ad71-4a59-ab1b-5be46cdc948e None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.088703 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7a22efa7-213c-4608-bf64-390132f06598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7035a32bcd241f7b46587e1967aaa8c', 'request_id': 'req-7a22efa7-213c-4608-bf64-390132f06598'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:50.088826 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448a665-ad71-4a59-ab1b-5be46cdc948e None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.089121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448a665-ad71-4a59-ab1b-5be46cdc948e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.102258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7a22efa7-213c-4608-bf64-390132f06598 admin admin] The token cache is being invalidate because project a7035a32bcd241f7b46587e1967aaa8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:25:50.102540 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7a22efa7-213c-4608-bf64-390132f06598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:50.103203 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7a22efa7-213c-4608-bf64-390132f06598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:25:50.105163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1042/2095] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:25:49 2022] DELETE /identity/v3/projects/a7035a32bcd241f7b46587e1967aaa8c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:50.108753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-d448a665-ad71-4a59-ab1b-5be46cdc948e None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:50.109799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1937f-7899-4d95-b746-32fa480e5020 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.110089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1937f-7899-4d95-b746-32fa480e5020 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.110193 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d448a665-ad71-4a59-ab1b-5be46cdc948e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:50.110407 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1937f-7899-4d95-b746-32fa480e5020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.129414 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-bfd1937f-7899-4d95-b746-32fa480e5020 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:50.130797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bfd1937f-7899-4d95-b746-32fa480e5020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:50.159600 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:50.174216 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1054/2096] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:50.183628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36a42b-950f-4d6e-a01a-02c6d00a2623 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.183914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36a42b-950f-4d6e-a01a-02c6d00a2623 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.184192 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36a42b-950f-4d6e-a01a-02c6d00a2623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.198225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1043/2097] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:50.203774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-7b36a42b-950f-4d6e-a01a-02c6d00a2623 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:50.205225 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-7b36a42b-950f-4d6e-a01a-02c6d00a2623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:50.206183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01586c-10d9-4025-8c69-511acf4619d6 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.206465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01586c-10d9-4025-8c69-511acf4619d6 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.206737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01586c-10d9-4025-8c69-511acf4619d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.213912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1055/2098] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:50.217929 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36a42b-950f-4d6e-a01a-02c6d00a2623 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:50.227946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36a42b-950f-4d6e-a01a-02c6d00a2623 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.228222 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-8e01586c-10d9-4025-8c69-511acf4619d6 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:50.229920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8e01586c-10d9-4025-8c69-511acf4619d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:50.233078 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb0f3d-b860-46df-9c94-5f7819f3c90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.233701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb0f3d-b860-46df-9c94-5f7819f3c90d admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.234021 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb0f3d-b860-46df-9c94-5f7819f3c90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.234290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb0f3d-b860-46df-9c94-5f7819f3c90d admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.234985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb0f3d-b860-46df-9c94-5f7819f3c90d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:25:50.235282 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb0f3d-b860-46df-9c94-5f7819f3c90d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.236869 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb0f3d-b860-46df-9c94-5f7819f3c90d admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.238509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1044/2099] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:50.242949 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01586c-10d9-4025-8c69-511acf4619d6 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:50.249513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1056/2100] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:25:50.252506 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e01586c-10d9-4025-8c69-511acf4619d6 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.253934 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb0f3d-b860-46df-9c94-5f7819f3c90d admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:50.258498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528e6f5-5f1a-4285-9800-2a5717dc62d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.259115 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528e6f5-5f1a-4285-9800-2a5717dc62d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.259382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528e6f5-5f1a-4285-9800-2a5717dc62d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.259648 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b528e6f5-5f1a-4285-9800-2a5717dc62d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.260459 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528e6f5-5f1a-4285-9800-2a5717dc62d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:25:50.260764 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528e6f5-5f1a-4285-9800-2a5717dc62d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.262425 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b528e6f5-5f1a-4285-9800-2a5717dc62d6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.264078 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb0f3d-b860-46df-9c94-5f7819f3c90d admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.269344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc4dfa-fb72-40e3-b1c5-d5505af4e337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.269999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc4dfa-fb72-40e3-b1c5-d5505af4e337 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.270317 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc4dfa-fb72-40e3-b1c5-d5505af4e337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.270579 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc4dfa-fb72-40e3-b1c5-d5505af4e337 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.271379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc4dfa-fb72-40e3-b1c5-d5505af4e337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.273288 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc4dfa-fb72-40e3-b1c5-d5505af4e337 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.274979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1045/2101] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:25:50.279083 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528e6f5-5f1a-4285-9800-2a5717dc62d6 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:50.290905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b528e6f5-5f1a-4285-9800-2a5717dc62d6 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.297160 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4ef4f-eb7c-4cab-9656-c7960993becd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.299148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4ef4f-eb7c-4cab-9656-c7960993becd admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.299423 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4ef4f-eb7c-4cab-9656-c7960993becd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.299703 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4ef4f-eb7c-4cab-9656-c7960993becd admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.300366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c4ef4f-eb7c-4cab-9656-c7960993becd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.302417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c4ef4f-eb7c-4cab-9656-c7960993becd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.309100 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1057/2102] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:25:50 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:50.310124 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc4dfa-fb72-40e3-b1c5-d5505af4e337 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:50.337442 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1046/2103] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:25:50 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:25:50.342029 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4ef4f-eb7c-4cab-9656-c7960993becd admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:50.384275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc4dfa-fb72-40e3-b1c5-d5505af4e337 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.396916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:50.415362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4ef4f-eb7c-4cab-9656-c7960993becd admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.421253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758ca97-3877-4dbd-b010-92f583ede82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.421934 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758ca97-3877-4dbd-b010-92f583ede82d admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.422210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758ca97-3877-4dbd-b010-92f583ede82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.422458 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758ca97-3877-4dbd-b010-92f583ede82d admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.423165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758ca97-3877-4dbd-b010-92f583ede82d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.425340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758ca97-3877-4dbd-b010-92f583ede82d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.452449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1047/2104] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:25:50 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:50.453539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758ca97-3877-4dbd-b010-92f583ede82d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:50.462160 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02503520-efc6-4b7f-8bd0-c401f21885a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.462782 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02503520-efc6-4b7f-8bd0-c401f21885a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.463057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02503520-efc6-4b7f-8bd0-c401f21885a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.463592 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02503520-efc6-4b7f-8bd0-c401f21885a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.464260 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02503520-efc6-4b7f-8bd0-c401f21885a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.464363 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758ca97-3877-4dbd-b010-92f583ede82d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.466216 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02503520-efc6-4b7f-8bd0-c401f21885a4 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.469326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c2bde-3a17-4156-b2b4-eadb06ccab25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.469967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c2bde-3a17-4156-b2b4-eadb06ccab25 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.470253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c2bde-3a17-4156-b2b4-eadb06ccab25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.470528 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c2bde-3a17-4156-b2b4-eadb06ccab25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.471275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c2bde-3a17-4156-b2b4-eadb06ccab25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.472877 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c2bde-3a17-4156-b2b4-eadb06ccab25 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.484758 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1048/2105] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:50.487585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1058/2106] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:25:50 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:50.489221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c2bde-3a17-4156-b2b4-eadb06ccab25 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:50.491887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02503520-efc6-4b7f-8bd0-c401f21885a4 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:50.499168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c2bde-3a17-4156-b2b4-eadb06ccab25 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.501480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02503520-efc6-4b7f-8bd0-c401f21885a4 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.504352 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6be97-1c62-48c2-b50f-22d7b5135bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.504964 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6be97-1c62-48c2-b50f-22d7b5135bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.505336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6be97-1c62-48c2-b50f-22d7b5135bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.508057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a417acd-3b89-40c9-974c-f9e7515d1bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.508678 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a417acd-3b89-40c9-974c-f9e7515d1bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.508957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a417acd-3b89-40c9-974c-f9e7515d1bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.509230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a417acd-3b89-40c9-974c-f9e7515d1bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.509970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a417acd-3b89-40c9-974c-f9e7515d1bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.510135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6be97-1c62-48c2-b50f-22d7b5135bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.510935 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6be97-1c62-48c2-b50f-22d7b5135bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.511559 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a417acd-3b89-40c9-974c-f9e7515d1bb1 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.513001 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6be97-1c62-48c2-b50f-22d7b5135bb5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.522293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1059/2107] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:50.526328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a417acd-3b89-40c9-974c-f9e7515d1bb1 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:50.528239 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1049/2108] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:50.533634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6be97-1c62-48c2-b50f-22d7b5135bb5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:50.538747 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a417acd-3b89-40c9-974c-f9e7515d1bb1 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.543660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d33f00-35e1-484a-b78f-72c70b345e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.544357 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d33f00-35e1-484a-b78f-72c70b345e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.544618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6be97-1c62-48c2-b50f-22d7b5135bb5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.544798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d33f00-35e1-484a-b78f-72c70b345e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.545122 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d33f00-35e1-484a-b78f-72c70b345e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.546000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d33f00-35e1-484a-b78f-72c70b345e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.547695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d33f00-35e1-484a-b78f-72c70b345e12 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.551610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1260a3-3601-4705-a6f8-707db1ecbffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.552305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1260a3-3601-4705-a6f8-707db1ecbffd admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.552586 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1260a3-3601-4705-a6f8-707db1ecbffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.552897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1260a3-3601-4705-a6f8-707db1ecbffd admin admin] PATH_INFO: `/v3/projects/b459ddf5e8724695af7357b8da059570/users/16d9d751f5ab4fff9b15f0a6a893d517/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.559656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1060/2109] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:25:50.563504 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d33f00-35e1-484a-b78f-72c70b345e12 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:50.569212 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1260a3-3601-4705-a6f8-707db1ecbffd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b459ddf5e8724695af7357b8da059570, user_id=16d9d751f5ab4fff9b15f0a6a893d517, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.572193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1260a3-3601-4705-a6f8-707db1ecbffd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.574763 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d33f00-35e1-484a-b78f-72c70b345e12 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.580985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa6ffb-f2ad-4c6b-ae15-b1e59a8a410a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.581672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa6ffb-f2ad-4c6b-ae15-b1e59a8a410a admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.582159 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa6ffb-f2ad-4c6b-ae15-b1e59a8a410a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.582267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa6ffb-f2ad-4c6b-ae15-b1e59a8a410a admin admin] PATH_INFO: `/v3/projects/3e412cf0f7f44299b12b59a8cf62f424/users/153507f4e7134d6a8ab87390326d4195/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.588686 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1050/2110] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:50 2022] PUT /identity/v3/projects/b459ddf5e8724695af7357b8da059570/users/16d9d751f5ab4fff9b15f0a6a893d517/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:50.598378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa6ffb-f2ad-4c6b-ae15-b1e59a8a410a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e412cf0f7f44299b12b59a8cf62f424, user_id=153507f4e7134d6a8ab87390326d4195, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.599823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227b0e3-bbce-4e87-93eb-bb519317f083 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.601100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227b0e3-bbce-4e87-93eb-bb519317f083 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.601100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7227b0e3-bbce-4e87-93eb-bb519317f083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.602053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa6ffb-f2ad-4c6b-ae15-b1e59a8a410a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.627139 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1061/2111] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:50 2022] PUT /identity/v3/projects/3e412cf0f7f44299b12b59a8cf62f424/users/153507f4e7134d6a8ab87390326d4195/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:50.631911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7227b0e3-bbce-4e87-93eb-bb519317f083 None None] MFA Rules not processed for user `16d9d751f5ab4fff9b15f0a6a893d517`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:50.632152 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba66af-bb76-49b2-93a2-94b84f482731 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.632493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba66af-bb76-49b2-93a2-94b84f482731 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.632750 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba66af-bb76-49b2-93a2-94b84f482731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.633457 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7227b0e3-bbce-4e87-93eb-bb519317f083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:50.642373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:50.658488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-74ba66af-bb76-49b2-93a2-94b84f482731 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:50.660242 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-74ba66af-bb76-49b2-93a2-94b84f482731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:50.703188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1051/2112] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:50.715669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227b0e3-bbce-4e87-93eb-bb519317f083 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:50.717106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7227b0e3-bbce-4e87-93eb-bb519317f083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:50.728151 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1062/2113] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:25:50.737602 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba66af-bb76-49b2-93a2-94b84f482731 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:50.739100 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-74ba66af-bb76-49b2-93a2-94b84f482731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:50.774905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7227b0e3-bbce-4e87-93eb-bb519317f083 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.778984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a796d39-d74b-447b-901b-027b9d87e2c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.779598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a796d39-d74b-447b-901b-027b9d87e2c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.779877 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a796d39-d74b-447b-901b-027b9d87e2c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.780146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a796d39-d74b-447b-901b-027b9d87e2c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.789586 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a796d39-d74b-447b-901b-027b9d87e2c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.791103 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a796d39-d74b-447b-901b-027b9d87e2c0 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.795074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba66af-bb76-49b2-93a2-94b84f482731 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.799624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a48eac-5a7b-4be9-a95b-cefce205b72d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.800281 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a48eac-5a7b-4be9-a95b-cefce205b72d service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.800525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1052/2114] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:50.800659 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a48eac-5a7b-4be9-a95b-cefce205b72d service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.800823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a48eac-5a7b-4be9-a95b-cefce205b72d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.810758 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a48eac-5a7b-4be9-a95b-cefce205b72d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.812187 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a48eac-5a7b-4be9-a95b-cefce205b72d service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.822909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1063/2115] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:51.648665 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552e2a6-89a4-4908-964b-2b891ad29b0f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:51.649169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552e2a6-89a4-4908-964b-2b891ad29b0f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:51.649260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552e2a6-89a4-4908-964b-2b891ad29b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:51.687967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5552e2a6-89a4-4908-964b-2b891ad29b0f None None] MFA Rules not processed for user `153507f4e7134d6a8ab87390326d4195`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:51.690339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5552e2a6-89a4-4908-964b-2b891ad29b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:51.706385 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:51.816666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1053/2116] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:51 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:51.822267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a48eac-5a7b-4be9-a95b-cefce205b72d service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:51.833819 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a48eac-5a7b-4be9-a95b-cefce205b72d service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:51.839269 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e6714-c05c-47df-a40b-cf6e92e449ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:51.839779 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e6714-c05c-47df-a40b-cf6e92e449ce admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:51.840269 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e6714-c05c-47df-a40b-cf6e92e449ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:51.840911 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e6714-c05c-47df-a40b-cf6e92e449ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:51.842018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e6714-c05c-47df-a40b-cf6e92e449ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:51.844401 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e6714-c05c-47df-a40b-cf6e92e449ce admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:51.888643 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1064/2117] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:25:51 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:25:51.889954 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552e2a6-89a4-4908-964b-2b891ad29b0f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:51.901626 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552e2a6-89a4-4908-964b-2b891ad29b0f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:51.958686 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7b13b-839b-4a52-95a2-c623b61c0b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:51.959311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7b13b-839b-4a52-95a2-c623b61c0b45 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:51.959595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7b13b-839b-4a52-95a2-c623b61c0b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:51.959869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7b13b-839b-4a52-95a2-c623b61c0b45 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:51.960535 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7b13b-839b-4a52-95a2-c623b61c0b45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:51.962838 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7b13b-839b-4a52-95a2-c623b61c0b45 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:51.987113 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1054/2118] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:25:51 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:25:51.991957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e6714-c05c-47df-a40b-cf6e92e449ce admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:52.003367 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e6714-c05c-47df-a40b-cf6e92e449ce admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:52.009011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45cd3a-7993-4ee4-9347-510ce0cd8015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:52.009627 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45cd3a-7993-4ee4-9347-510ce0cd8015 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:52.009940 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45cd3a-7993-4ee4-9347-510ce0cd8015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:52.010219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45cd3a-7993-4ee4-9347-510ce0cd8015 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:52.010924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b45cd3a-7993-4ee4-9347-510ce0cd8015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:52.012529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b45cd3a-7993-4ee4-9347-510ce0cd8015 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:52.022878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1065/2119] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:25:52.027214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7b13b-839b-4a52-95a2-c623b61c0b45 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:52.037926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7b13b-839b-4a52-95a2-c623b61c0b45 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:52.044303 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26dfbd-dd05-458a-9908-f032856cbdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:52.044974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26dfbd-dd05-458a-9908-f032856cbdef admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:52.045216 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26dfbd-dd05-458a-9908-f032856cbdef admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:52.045512 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26dfbd-dd05-458a-9908-f032856cbdef admin admin] PATH_INFO: `/v3/projects/ccce5c49cf2549dd94d62cb36c67cab2/users/9b66ddf9d9104cfaa93f7fbab3bae30c/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:52.063139 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26dfbd-dd05-458a-9908-f032856cbdef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccce5c49cf2549dd94d62cb36c67cab2, user_id=9b66ddf9d9104cfaa93f7fbab3bae30c, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:52.065864 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26dfbd-dd05-458a-9908-f032856cbdef admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:52.080500 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1055/2120] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:52 2022] PUT /identity/v3/projects/ccce5c49cf2549dd94d62cb36c67cab2/users/9b66ddf9d9104cfaa93f7fbab3bae30c/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:25:52.084662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45cd3a-7993-4ee4-9347-510ce0cd8015 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:52.096909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45cd3a-7993-4ee4-9347-510ce0cd8015 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:52.156077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfa90f-0149-4be7-ada5-da68ce299de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:52.156702 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfa90f-0149-4be7-ada5-da68ce299de5 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:52.157092 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfa90f-0149-4be7-ada5-da68ce299de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:52.157396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfa90f-0149-4be7-ada5-da68ce299de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:52.158162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfa90f-0149-4be7-ada5-da68ce299de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:52.159726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dfa90f-0149-4be7-ada5-da68ce299de5 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:52.171019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1066/2121] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:25:52.175539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26dfbd-dd05-458a-9908-f032856cbdef admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:52.189894 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26dfbd-dd05-458a-9908-f032856cbdef admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:52.196076 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479aea34-e23c-4a74-b2c7-ca6e95109545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:52.196761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479aea34-e23c-4a74-b2c7-ca6e95109545 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:52.197039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479aea34-e23c-4a74-b2c7-ca6e95109545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:52.197308 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479aea34-e23c-4a74-b2c7-ca6e95109545 admin admin] PATH_INFO: `/v3/projects/ccce5c49cf2549dd94d62cb36c67cab2/users/9b66ddf9d9104cfaa93f7fbab3bae30c/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:52.199479 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479aea34-e23c-4a74-b2c7-ca6e95109545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccce5c49cf2549dd94d62cb36c67cab2, user_id=9b66ddf9d9104cfaa93f7fbab3bae30c, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:52.203114 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479aea34-e23c-4a74-b2c7-ca6e95109545 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:52.220921 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1056/2122] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:25:52 2022] PUT /identity/v3/projects/ccce5c49cf2549dd94d62cb36c67cab2/users/9b66ddf9d9104cfaa93f7fbab3bae30c/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:25:53.214579 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340783ad-67f1-42fe-abc3-85d7fe83580b None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:53.215182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340783ad-67f1-42fe-abc3-85d7fe83580b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:53.215182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340783ad-67f1-42fe-abc3-85d7fe83580b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:53.243172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:53.263895 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-340783ad-67f1-42fe-abc3-85d7fe83580b None None] MFA Rules not processed for user `9b66ddf9d9104cfaa93f7fbab3bae30c`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:53.265217 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-340783ad-67f1-42fe-abc3-85d7fe83580b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:53.380459 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1067/2123] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:53.388676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479aea34-e23c-4a74-b2c7-ca6e95109545 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:53.478943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479aea34-e23c-4a74-b2c7-ca6e95109545 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:53.483121 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649054df-f06c-4171-88a5-18d6063669ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:53.485928 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649054df-f06c-4171-88a5-18d6063669ea service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:53.486328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649054df-f06c-4171-88a5-18d6063669ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:53.486705 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649054df-f06c-4171-88a5-18d6063669ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:53.500193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:53.555406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649054df-f06c-4171-88a5-18d6063669ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:53.557223 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649054df-f06c-4171-88a5-18d6063669ea service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:53.568408 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1057/2124] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:53 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:57.070135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340783ad-67f1-42fe-abc3-85d7fe83580b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:57.080242 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340783ad-67f1-42fe-abc3-85d7fe83580b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:57.084722 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb86b7-2443-4330-b745-40adefceabe2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:57.085338 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb86b7-2443-4330-b745-40adefceabe2 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:57.088128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb86b7-2443-4330-b745-40adefceabe2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:57.088128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb86b7-2443-4330-b745-40adefceabe2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:57.096916 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb86b7-2443-4330-b745-40adefceabe2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:57.098413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb86b7-2443-4330-b745-40adefceabe2 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:57.111009 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1068/2125] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:25:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:58.139876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ec5b2-4439-4f52-8de6-58b7d6795041 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:58.139876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ec5b2-4439-4f52-8de6-58b7d6795041 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:58.139876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ec5b2-4439-4f52-8de6-58b7d6795041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:58.157242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-696ec5b2-4439-4f52-8de6-58b7d6795041 None None] MFA Rules not processed for user `6f852249c66a4f8986d7a63a957ec91c`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:58.158578 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-696ec5b2-4439-4f52-8de6-58b7d6795041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:58.219788 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1058/2126] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:25:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:07.105461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=36 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:08.212605 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=45 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:08.674143 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb86b7-2443-4330-b745-40adefceabe2 service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:08.773707 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb86b7-2443-4330-b745-40adefceabe2 service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:08.777586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d460d-2e48-4654-a7f7-314ce15223d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:08.778267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d460d-2e48-4654-a7f7-314ce15223d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:08.778544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d460d-2e48-4654-a7f7-314ce15223d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:08.778804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d460d-2e48-4654-a7f7-314ce15223d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:08.780729 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8d0d460d-2e48-4654-a7f7-314ce15223d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:08.859876 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d460d-2e48-4654-a7f7-314ce15223d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:08.862078 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d460d-2e48-4654-a7f7-314ce15223d9 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:08.876128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1069/2127] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:12.640783 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ec5b2-4439-4f52-8de6-58b7d6795041 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:12.743555 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ec5b2-4439-4f52-8de6-58b7d6795041 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:12.749019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8ceba-9e92-4028-99e4-0f23edcbfa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:12.749952 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8ceba-9e92-4028-99e4-0f23edcbfa23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:12.750457 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8ceba-9e92-4028-99e4-0f23edcbfa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:12.751210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8ceba-9e92-4028-99e4-0f23edcbfa23 admin admin] PATH_INFO: `/v3/users/16d9d751f5ab4fff9b15f0a6a893d517` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:12.752969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8ceba-9e92-4028-99e4-0f23edcbfa23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16d9d751f5ab4fff9b15f0a6a893d517)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:12.755213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8ceba-9e92-4028-99e4-0f23edcbfa23 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:12.820180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6bd8ceba-9e92-4028-99e4-0f23edcbfa23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16d9d751f5ab4fff9b15f0a6a893d517'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:12.823188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6bd8ceba-9e92-4028-99e4-0f23edcbfa23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16d9d751f5ab4fff9b15f0a6a893d517'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:12.846968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6bd8ceba-9e92-4028-99e4-0f23edcbfa23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16d9d751f5ab4fff9b15f0a6a893d517'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:12.866050 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6bd8ceba-9e92-4028-99e4-0f23edcbfa23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16d9d751f5ab4fff9b15f0a6a893d517'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:12.877969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6bd8ceba-9e92-4028-99e4-0f23edcbfa23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16d9d751f5ab4fff9b15f0a6a893d517'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:12.893845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1059/2128] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:26:12 2022] DELETE /identity/v3/users/16d9d751f5ab4fff9b15f0a6a893d517 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:12.898893 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975eab1-df8e-4916-ab0d-61b39f909766 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:12.899399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975eab1-df8e-4916-ab0d-61b39f909766 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:12.899514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975eab1-df8e-4916-ab0d-61b39f909766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:12.919563 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-9975eab1-df8e-4916-ab0d-61b39f909766 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:12.920929 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9975eab1-df8e-4916-ab0d-61b39f909766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:13.001080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1070/2129] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:13.012568 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8ceba-9e92-4028-99e4-0f23edcbfa23 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:13.014397 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6bd8ceba-9e92-4028-99e4-0f23edcbfa23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:13.082319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8ceba-9e92-4028-99e4-0f23edcbfa23 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:13.086889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72e689-3f38-47c7-b6bc-fe7005fc01cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:13.087564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72e689-3f38-47c7-b6bc-fe7005fc01cd service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:13.087842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72e689-3f38-47c7-b6bc-fe7005fc01cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:13.088113 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72e689-3f38-47c7-b6bc-fe7005fc01cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:13.100074 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72e689-3f38-47c7-b6bc-fe7005fc01cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:13.101584 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72e689-3f38-47c7-b6bc-fe7005fc01cd service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:13.113345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1060/2130] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:13.527655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975eab1-df8e-4916-ab0d-61b39f909766 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:13.529129 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9975eab1-df8e-4916-ab0d-61b39f909766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:13.541533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975eab1-df8e-4916-ab0d-61b39f909766 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:13.547346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a4e9a-5e0c-4b93-b010-2c14a4ff066c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:13.548076 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a4e9a-5e0c-4b93-b010-2c14a4ff066c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:13.548426 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a4e9a-5e0c-4b93-b010-2c14a4ff066c admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:13.548771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a4e9a-5e0c-4b93-b010-2c14a4ff066c admin admin] PATH_INFO: `/v3/projects/b459ddf5e8724695af7357b8da059570` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:13.550351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a4e9a-5e0c-4b93-b010-2c14a4ff066c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b459ddf5e8724695af7357b8da059570)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:13.552437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a4e9a-5e0c-4b93-b010-2c14a4ff066c admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:13.603210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:13.606140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-185a4e9a-5e0c-4b93-b010-2c14a4ff066c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b459ddf5e8724695af7357b8da059570', 'request_id': 'req-185a4e9a-5e0c-4b93-b010-2c14a4ff066c'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:13.615664 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-185a4e9a-5e0c-4b93-b010-2c14a4ff066c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b459ddf5e8724695af7357b8da059570', 'request_id': 'req-185a4e9a-5e0c-4b93-b010-2c14a4ff066c'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:13.623742 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-185a4e9a-5e0c-4b93-b010-2c14a4ff066c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b459ddf5e8724695af7357b8da059570', 'request_id': 'req-185a4e9a-5e0c-4b93-b010-2c14a4ff066c'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:13.638351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-185a4e9a-5e0c-4b93-b010-2c14a4ff066c admin admin] The token cache is being invalidate because project b459ddf5e8724695af7357b8da059570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:13.638640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-185a4e9a-5e0c-4b93-b010-2c14a4ff066c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:13.639320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-185a4e9a-5e0c-4b93-b010-2c14a4ff066c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:13.641118 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1071/2131] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:26:13 2022] DELETE /identity/v3/projects/b459ddf5e8724695af7357b8da059570 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:13.646859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63d3f3-635d-43a6-8e6c-69aa436a0736 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:13.647412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63d3f3-635d-43a6-8e6c-69aa436a0736 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:13.647412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63d3f3-635d-43a6-8e6c-69aa436a0736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:13.668037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4f63d3f3-635d-43a6-8e6c-69aa436a0736 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:13.669248 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4f63d3f3-635d-43a6-8e6c-69aa436a0736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:13.687385 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:13.756641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1061/2132] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:13.764696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee272b-9a34-4270-90f6-09244bb017f3 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:13.765005 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee272b-9a34-4270-90f6-09244bb017f3 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:13.765280 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee272b-9a34-4270-90f6-09244bb017f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:13.785210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-26ee272b-9a34-4270-90f6-09244bb017f3 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:13.786482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-26ee272b-9a34-4270-90f6-09244bb017f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:13.795238 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1072/2133] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:13.799650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63d3f3-635d-43a6-8e6c-69aa436a0736 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:13.809712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63d3f3-635d-43a6-8e6c-69aa436a0736 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:13.814816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e09cf-7ef9-449b-9b48-4f5836e66ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:13.815450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e09cf-7ef9-449b-9b48-4f5836e66ccd admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:13.815729 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e09cf-7ef9-449b-9b48-4f5836e66ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:13.816005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e09cf-7ef9-449b-9b48-4f5836e66ccd admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:13.816733 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e09cf-7ef9-449b-9b48-4f5836e66ccd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:26:13.817013 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e09cf-7ef9-449b-9b48-4f5836e66ccd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:13.818704 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36e09cf-7ef9-449b-9b48-4f5836e66ccd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:13.835571 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1062/2134] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:26:13.837182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee272b-9a34-4270-90f6-09244bb017f3 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:13.848694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee272b-9a34-4270-90f6-09244bb017f3 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:13.854472 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39b49f-1d32-4464-bcb2-59fc805aa28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:13.855116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39b49f-1d32-4464-bcb2-59fc805aa28f admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:13.855393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39b49f-1d32-4464-bcb2-59fc805aa28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:13.855666 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39b49f-1d32-4464-bcb2-59fc805aa28f admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:13.856376 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39b49f-1d32-4464-bcb2-59fc805aa28f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:13.858695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39b49f-1d32-4464-bcb2-59fc805aa28f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:13.908195 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1073/2135] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:26:13 2022] POST /identity/v3/projects => generated 369 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:13.912659 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e09cf-7ef9-449b-9b48-4f5836e66ccd admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:13.923909 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36e09cf-7ef9-449b-9b48-4f5836e66ccd admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:14.009700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71198847-bb2b-4e66-bfe2-395dc12abb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:14.010381 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71198847-bb2b-4e66-bfe2-395dc12abb46 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:14.010663 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71198847-bb2b-4e66-bfe2-395dc12abb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:14.010938 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71198847-bb2b-4e66-bfe2-395dc12abb46 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:14.011691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71198847-bb2b-4e66-bfe2-395dc12abb46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:14.014001 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71198847-bb2b-4e66-bfe2-395dc12abb46 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:14.041534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1063/2136] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:26:13 2022] POST /identity/v3/users => generated 285 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:14.045456 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39b49f-1d32-4464-bcb2-59fc805aa28f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:14.058153 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39b49f-1d32-4464-bcb2-59fc805aa28f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:14.063700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b26980-fe5a-428a-9741-0aeb2b632cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:14.064346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b26980-fe5a-428a-9741-0aeb2b632cec admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:14.064626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b26980-fe5a-428a-9741-0aeb2b632cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:14.064906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b26980-fe5a-428a-9741-0aeb2b632cec admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:14.065628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b26980-fe5a-428a-9741-0aeb2b632cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:14.067355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b26980-fe5a-428a-9741-0aeb2b632cec admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:14.079098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1074/2137] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:14.083304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71198847-bb2b-4e66-bfe2-395dc12abb46 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:14.099582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71198847-bb2b-4e66-bfe2-395dc12abb46 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:14.104831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c17d4-e2c4-4054-90a9-4180ec938dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:14.105590 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c17d4-e2c4-4054-90a9-4180ec938dbd admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:14.106016 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c17d4-e2c4-4054-90a9-4180ec938dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:14.106389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c17d4-e2c4-4054-90a9-4180ec938dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:14.107178 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c17d4-e2c4-4054-90a9-4180ec938dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:14.109177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c17d4-e2c4-4054-90a9-4180ec938dbd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:14.125136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1064/2138] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:14.129462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b26980-fe5a-428a-9741-0aeb2b632cec admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:14.149836 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b26980-fe5a-428a-9741-0aeb2b632cec admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:14.155303 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38aa7b-bd63-4760-9c12-19e37feb43d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:14.156307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38aa7b-bd63-4760-9c12-19e37feb43d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:14.156587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38aa7b-bd63-4760-9c12-19e37feb43d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:14.156858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38aa7b-bd63-4760-9c12-19e37feb43d9 admin admin] PATH_INFO: `/v3/projects/d00affda994040aeb2654ade7575502b/users/edfe17b405664e96bbeb149dc9c38378/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:14.177554 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38aa7b-bd63-4760-9c12-19e37feb43d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d00affda994040aeb2654ade7575502b, user_id=edfe17b405664e96bbeb149dc9c38378, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:14.180279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38aa7b-bd63-4760-9c12-19e37feb43d9 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:14.197270 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1075/2139] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:14 2022] PUT /identity/v3/projects/d00affda994040aeb2654ade7575502b/users/edfe17b405664e96bbeb149dc9c38378/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:14.209867 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290bb50-b461-4965-b01b-96a9e9b9a4e2 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:14.213878 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290bb50-b461-4965-b01b-96a9e9b9a4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:14.213878 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290bb50-b461-4965-b01b-96a9e9b9a4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:14.247367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e290bb50-b461-4965-b01b-96a9e9b9a4e2 None None] MFA Rules not processed for user `edfe17b405664e96bbeb149dc9c38378`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:14.248929 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e290bb50-b461-4965-b01b-96a9e9b9a4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:14.341816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1065/2140] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:14 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:14.352720 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38aa7b-bd63-4760-9c12-19e37feb43d9 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:14.354732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6f38aa7b-bd63-4760-9c12-19e37feb43d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:14.440058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38aa7b-bd63-4760-9c12-19e37feb43d9 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:14.443744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839134a2-cb6c-4585-9b85-6579e3cae0d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f696dfe4b34c4aa19104b5f7ee6e70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:14.444372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839134a2-cb6c-4585-9b85-6579e3cae0d7 service glance] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:14.444650 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839134a2-cb6c-4585-9b85-6579e3cae0d7 service glance] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:14.444920 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839134a2-cb6c-4585-9b85-6579e3cae0d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:14.453907 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839134a2-cb6c-4585-9b85-6579e3cae0d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:14.455361 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839134a2-cb6c-4585-9b85-6579e3cae0d7 service glance] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:14.464764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1076/2141] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:14 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:16.969771 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290bb50-b461-4965-b01b-96a9e9b9a4e2 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:16.971263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e290bb50-b461-4965-b01b-96a9e9b9a4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:17.000463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:17.066030 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290bb50-b461-4965-b01b-96a9e9b9a4e2 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:17.074903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d975bb6-8e12-4dd8-8796-b11255df6e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:17.075566 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d975bb6-8e12-4dd8-8796-b11255df6e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:17.076358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d975bb6-8e12-4dd8-8796-b11255df6e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:17.077755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d975bb6-8e12-4dd8-8796-b11255df6e7d admin admin] PATH_INFO: `/v3/users/153507f4e7134d6a8ab87390326d4195` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:17.078027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d975bb6-8e12-4dd8-8796-b11255df6e7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=153507f4e7134d6a8ab87390326d4195)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:17.081683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d975bb6-8e12-4dd8-8796-b11255df6e7d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:17.150784 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5d975bb6-8e12-4dd8-8796-b11255df6e7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '153507f4e7134d6a8ab87390326d4195'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:17.151584 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5d975bb6-8e12-4dd8-8796-b11255df6e7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '153507f4e7134d6a8ab87390326d4195'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:17.169099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5d975bb6-8e12-4dd8-8796-b11255df6e7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '153507f4e7134d6a8ab87390326d4195'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:17.179049 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5d975bb6-8e12-4dd8-8796-b11255df6e7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '153507f4e7134d6a8ab87390326d4195'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:17.186582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5d975bb6-8e12-4dd8-8796-b11255df6e7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '153507f4e7134d6a8ab87390326d4195'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:17.196811 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1066/2142] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:26:16 2022] DELETE /identity/v3/users/153507f4e7134d6a8ab87390326d4195 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:17.401913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839134a2-cb6c-4585-9b85-6579e3cae0d7 service glance] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:17.422400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839134a2-cb6c-4585-9b85-6579e3cae0d7 service glance] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:17.494047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:17.511603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd832a-037c-45d5-8179-1233ea77e285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:17.513799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd832a-037c-45d5-8179-1233ea77e285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:17.514428 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd832a-037c-45d5-8179-1233ea77e285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:17.515078 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd832a-037c-45d5-8179-1233ea77e285 admin admin] PATH_INFO: `/v3/projects/3e412cf0f7f44299b12b59a8cf62f424` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:17.518040 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd832a-037c-45d5-8179-1233ea77e285 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e412cf0f7f44299b12b59a8cf62f424)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:17.521530 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd832a-037c-45d5-8179-1233ea77e285 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:17.588826 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-adfd832a-037c-45d5-8179-1233ea77e285 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e412cf0f7f44299b12b59a8cf62f424', 'request_id': 'req-adfd832a-037c-45d5-8179-1233ea77e285'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:17.610395 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-adfd832a-037c-45d5-8179-1233ea77e285 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e412cf0f7f44299b12b59a8cf62f424', 'request_id': 'req-adfd832a-037c-45d5-8179-1233ea77e285'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:17.619498 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-adfd832a-037c-45d5-8179-1233ea77e285 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e412cf0f7f44299b12b59a8cf62f424', 'request_id': 'req-adfd832a-037c-45d5-8179-1233ea77e285'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:17.636203 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-adfd832a-037c-45d5-8179-1233ea77e285 admin admin] The token cache is being invalidate because project 3e412cf0f7f44299b12b59a8cf62f424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:17.637055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-adfd832a-037c-45d5-8179-1233ea77e285 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:17.637488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-adfd832a-037c-45d5-8179-1233ea77e285 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:17.639906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1077/2143] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:26:17 2022] DELETE /identity/v3/projects/3e412cf0f7f44299b12b59a8cf62f424 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:17.644906 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d975bb6-8e12-4dd8-8796-b11255df6e7d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:17.647503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5d975bb6-8e12-4dd8-8796-b11255df6e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:17.678744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d975bb6-8e12-4dd8-8796-b11255df6e7d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:17.836755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd97bac-1297-47df-ac4c-a753be79ef82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:17.837522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd97bac-1297-47df-ac4c-a753be79ef82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:17.838013 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd97bac-1297-47df-ac4c-a753be79ef82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:17.838291 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd97bac-1297-47df-ac4c-a753be79ef82 admin admin] PATH_INFO: `/v3/users/9b66ddf9d9104cfaa93f7fbab3bae30c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:17.840513 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd97bac-1297-47df-ac4c-a753be79ef82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b66ddf9d9104cfaa93f7fbab3bae30c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:17.842674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd97bac-1297-47df-ac4c-a753be79ef82 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:17.918690 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:17.929229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-bcd97bac-1297-47df-ac4c-a753be79ef82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b66ddf9d9104cfaa93f7fbab3bae30c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:17.932089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-bcd97bac-1297-47df-ac4c-a753be79ef82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b66ddf9d9104cfaa93f7fbab3bae30c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:17.949640 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-bcd97bac-1297-47df-ac4c-a753be79ef82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b66ddf9d9104cfaa93f7fbab3bae30c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:17.965469 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-bcd97bac-1297-47df-ac4c-a753be79ef82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b66ddf9d9104cfaa93f7fbab3bae30c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:17.973993 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-bcd97bac-1297-47df-ac4c-a753be79ef82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b66ddf9d9104cfaa93f7fbab3bae30c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:17.981952 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1067/2144] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:26:17 2022] DELETE /identity/v3/users/9b66ddf9d9104cfaa93f7fbab3bae30c => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:18.229391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd832a-037c-45d5-8179-1233ea77e285 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:18.231375 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-adfd832a-037c-45d5-8179-1233ea77e285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:18.246224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd832a-037c-45d5-8179-1233ea77e285 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:18.343637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a99e7-2808-4db9-9056-ae1cb4550ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:18.344935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a99e7-2808-4db9-9056-ae1cb4550ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:18.344935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a99e7-2808-4db9-9056-ae1cb4550ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:18.345193 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a99e7-2808-4db9-9056-ae1cb4550ef7 admin admin] PATH_INFO: `/v3/projects/ccce5c49cf2549dd94d62cb36c67cab2` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:18.353828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a99e7-2808-4db9-9056-ae1cb4550ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccce5c49cf2549dd94d62cb36c67cab2)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:18.355820 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a99e7-2808-4db9-9056-ae1cb4550ef7 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:18.435743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ed6a99e7-2808-4db9-9056-ae1cb4550ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccce5c49cf2549dd94d62cb36c67cab2', 'request_id': 'req-ed6a99e7-2808-4db9-9056-ae1cb4550ef7'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:18.447205 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ed6a99e7-2808-4db9-9056-ae1cb4550ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccce5c49cf2549dd94d62cb36c67cab2', 'request_id': 'req-ed6a99e7-2808-4db9-9056-ae1cb4550ef7'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:18.455177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ed6a99e7-2808-4db9-9056-ae1cb4550ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccce5c49cf2549dd94d62cb36c67cab2', 'request_id': 'req-ed6a99e7-2808-4db9-9056-ae1cb4550ef7'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:18.469910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ed6a99e7-2808-4db9-9056-ae1cb4550ef7 admin admin] The token cache is being invalidate because project ccce5c49cf2549dd94d62cb36c67cab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:18.470193 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ed6a99e7-2808-4db9-9056-ae1cb4550ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:18.470840 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ed6a99e7-2808-4db9-9056-ae1cb4550ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:18.473547 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1078/2145] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:26:18 2022] DELETE /identity/v3/projects/ccce5c49cf2549dd94d62cb36c67cab2 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:18.481107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f7e68-486c-4bda-a056-55304dd8d88d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:18.481951 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f7e68-486c-4bda-a056-55304dd8d88d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:18.482405 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f7e68-486c-4bda-a056-55304dd8d88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:18.511430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-802f7e68-486c-4bda-a056-55304dd8d88d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:18.513427 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-802f7e68-486c-4bda-a056-55304dd8d88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:18.613946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1068/2146] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:18.623445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d70bfe-4310-4058-afc0-fc4aad475005 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:18.623945 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d70bfe-4310-4058-afc0-fc4aad475005 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:18.624504 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d70bfe-4310-4058-afc0-fc4aad475005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:18.646735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-b0d70bfe-4310-4058-afc0-fc4aad475005 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:18.648058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-b0d70bfe-4310-4058-afc0-fc4aad475005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:18.659087 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1079/2147] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:18.663137 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f7e68-486c-4bda-a056-55304dd8d88d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:18.677859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f7e68-486c-4bda-a056-55304dd8d88d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:18.679931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe1d3e-e89b-4806-88b5-13614c5354b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:18.680557 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe1d3e-e89b-4806-88b5-13614c5354b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:18.680835 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe1d3e-e89b-4806-88b5-13614c5354b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:18.681165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe1d3e-e89b-4806-88b5-13614c5354b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:18.682384 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe1d3e-e89b-4806-88b5-13614c5354b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:26:18.682384 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe1d3e-e89b-4806-88b5-13614c5354b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:18.684065 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbe1d3e-e89b-4806-88b5-13614c5354b3 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:18.704932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1069/2148] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:26:18.705383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d70bfe-4310-4058-afc0-fc4aad475005 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:18.717619 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d70bfe-4310-4058-afc0-fc4aad475005 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:18.724397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e2ae0-7b99-4559-948e-a8024b38c016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:18.725048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e2ae0-7b99-4559-948e-a8024b38c016 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:18.725773 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e2ae0-7b99-4559-948e-a8024b38c016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:18.725773 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e2ae0-7b99-4559-948e-a8024b38c016 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:18.726433 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e2ae0-7b99-4559-948e-a8024b38c016 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:18.728485 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e2ae0-7b99-4559-948e-a8024b38c016 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:18.733636 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:18.788227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe1d3e-e89b-4806-88b5-13614c5354b3 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:18.797942 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1080/2149] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:26:18 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:26:18.807447 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe1d3e-e89b-4806-88b5-13614c5354b3 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:18.890927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c29ecd-d1c2-43ea-a799-809c9e6203a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:18.891656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c29ecd-d1c2-43ea-a799-809c9e6203a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:18.891901 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c29ecd-d1c2-43ea-a799-809c9e6203a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:18.892325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c29ecd-d1c2-43ea-a799-809c9e6203a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:18.893013 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c29ecd-d1c2-43ea-a799-809c9e6203a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:18.895870 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c29ecd-d1c2-43ea-a799-809c9e6203a7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:18.921382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1070/2150] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:26:18 2022] POST /identity/v3/users => generated 288 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:26:18.926628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e2ae0-7b99-4559-948e-a8024b38c016 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:18.940046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e2ae0-7b99-4559-948e-a8024b38c016 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:18.947285 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffe955-2f37-4171-a11a-ecd37c5237a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:18.947865 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffe955-2f37-4171-a11a-ecd37c5237a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:18.948226 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffe955-2f37-4171-a11a-ecd37c5237a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:18.949099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ffe955-2f37-4171-a11a-ecd37c5237a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:18.949887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffe955-2f37-4171-a11a-ecd37c5237a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:18.951919 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ffe955-2f37-4171-a11a-ecd37c5237a4 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:18.974318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1081/2151] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:18.975330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c29ecd-d1c2-43ea-a799-809c9e6203a7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:18.987697 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c29ecd-d1c2-43ea-a799-809c9e6203a7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:19.000603 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b74d3-9aef-4641-816f-56ba90286fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:19.001270 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b74d3-9aef-4641-816f-56ba90286fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:19.001582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b74d3-9aef-4641-816f-56ba90286fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:19.001894 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b74d3-9aef-4641-816f-56ba90286fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:19.002620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b74d3-9aef-4641-816f-56ba90286fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:19.004289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b74d3-9aef-4641-816f-56ba90286fb5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:19.025582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1071/2152] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:19.031638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffe955-2f37-4171-a11a-ecd37c5237a4 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:19.044630 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ffe955-2f37-4171-a11a-ecd37c5237a4 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:19.051111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621d400-04b4-4f3e-870f-903957b204d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:19.052131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621d400-04b4-4f3e-870f-903957b204d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:19.052717 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621d400-04b4-4f3e-870f-903957b204d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:19.053290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621d400-04b4-4f3e-870f-903957b204d9 admin admin] PATH_INFO: `/v3/projects/15af566aad214a768abb25bc7bcc94b9/users/bce216aab8e34289b10ec3e0a9915e69/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:19.076671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621d400-04b4-4f3e-870f-903957b204d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15af566aad214a768abb25bc7bcc94b9, user_id=bce216aab8e34289b10ec3e0a9915e69, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:19.079811 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621d400-04b4-4f3e-870f-903957b204d9 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:19.097616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1082/2153] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:19 2022] PUT /identity/v3/projects/15af566aad214a768abb25bc7bcc94b9/users/bce216aab8e34289b10ec3e0a9915e69/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:19.108748 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0d4ea-2b63-4a00-ba4e-d1936883ced4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:19.109395 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0d4ea-2b63-4a00-ba4e-d1936883ced4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:19.110025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0d4ea-2b63-4a00-ba4e-d1936883ced4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:19.156135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2af0d4ea-2b63-4a00-ba4e-d1936883ced4 None None] MFA Rules not processed for user `bce216aab8e34289b10ec3e0a9915e69`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:19.157693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2af0d4ea-2b63-4a00-ba4e-d1936883ced4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:19.172721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:19.284101 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1072/2154] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:26:19.297251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621d400-04b4-4f3e-870f-903957b204d9 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:19.299294 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-a621d400-04b4-4f3e-870f-903957b204d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:19.378505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621d400-04b4-4f3e-870f-903957b204d9 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:19.384416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cad76-46ea-49aa-8709-c697d126362b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:19.384887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cad76-46ea-49aa-8709-c697d126362b service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:19.385169 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cad76-46ea-49aa-8709-c697d126362b service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:19.385443 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cad76-46ea-49aa-8709-c697d126362b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:19.396201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cad76-46ea-49aa-8709-c697d126362b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:19.397913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cad76-46ea-49aa-8709-c697d126362b service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:19.408749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1083/2155] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:20.104415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8a67-65ec-47b3-a764-4df13bcb29c4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:20.104954 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8a67-65ec-47b3-a764-4df13bcb29c4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:20.104954 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8a67-65ec-47b3-a764-4df13bcb29c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:20.125167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-db5f8a67-65ec-47b3-a764-4df13bcb29c4 None None] MFA Rules not processed for user `d79f95d9699b4b0393e14a9212986ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:20.126409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-db5f8a67-65ec-47b3-a764-4df13bcb29c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:20.204716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1073/2156] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:22.692636 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cad76-46ea-49aa-8709-c697d126362b service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:22.765695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cad76-46ea-49aa-8709-c697d126362b service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:22.771235 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a92c1e-99d1-4775-b1bd-cbc0f7c43fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:22.771987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a92c1e-99d1-4775-b1bd-cbc0f7c43fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:22.772292 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a92c1e-99d1-4775-b1bd-cbc0f7c43fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:22.772636 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a92c1e-99d1-4775-b1bd-cbc0f7c43fbe admin admin] PATH_INFO: `/v3/users/bce216aab8e34289b10ec3e0a9915e69` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:22.773752 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a92c1e-99d1-4775-b1bd-cbc0f7c43fbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=bce216aab8e34289b10ec3e0a9915e69)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:22.775864 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a92c1e-99d1-4775-b1bd-cbc0f7c43fbe admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:22.827352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-13a92c1e-99d1-4775-b1bd-cbc0f7c43fbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bce216aab8e34289b10ec3e0a9915e69'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:22.839278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-13a92c1e-99d1-4775-b1bd-cbc0f7c43fbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bce216aab8e34289b10ec3e0a9915e69'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:22.839985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-13a92c1e-99d1-4775-b1bd-cbc0f7c43fbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bce216aab8e34289b10ec3e0a9915e69'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:22.841641 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:22.850088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-13a92c1e-99d1-4775-b1bd-cbc0f7c43fbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bce216aab8e34289b10ec3e0a9915e69'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:22.865367 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-13a92c1e-99d1-4775-b1bd-cbc0f7c43fbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bce216aab8e34289b10ec3e0a9915e69'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:22.871839 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1084/2157] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:26:22 2022] DELETE /identity/v3/users/bce216aab8e34289b10ec3e0a9915e69 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:22.877540 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b65d5-3c04-4ef6-92bd-93f6bd34dbc3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:22.878900 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b65d5-3c04-4ef6-92bd-93f6bd34dbc3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:22.879245 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b65d5-3c04-4ef6-92bd-93f6bd34dbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:22.898912 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-893b65d5-3c04-4ef6-92bd-93f6bd34dbc3 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:22.901120 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-893b65d5-3c04-4ef6-92bd-93f6bd34dbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:22.980747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1074/2158] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:26:22.990710 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a92c1e-99d1-4775-b1bd-cbc0f7c43fbe admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:22.992218 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-13a92c1e-99d1-4775-b1bd-cbc0f7c43fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:23.065072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a92c1e-99d1-4775-b1bd-cbc0f7c43fbe admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:23.071306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca9773c-6875-415a-9cda-7745b23d9b3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:23.072315 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9773c-6875-415a-9cda-7745b23d9b3e service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:23.073689 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9773c-6875-415a-9cda-7745b23d9b3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:23.074327 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca9773c-6875-415a-9cda-7745b23d9b3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:23.087908 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9773c-6875-415a-9cda-7745b23d9b3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:23.089495 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca9773c-6875-415a-9cda-7745b23d9b3e service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:23.100245 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1085/2159] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:23.459571 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b65d5-3c04-4ef6-92bd-93f6bd34dbc3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:23.461670 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-893b65d5-3c04-4ef6-92bd-93f6bd34dbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:23.507332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b65d5-3c04-4ef6-92bd-93f6bd34dbc3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:23.521960 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb4943e-3f13-4c0a-8572-91d32edba874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:23.522791 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4943e-3f13-4c0a-8572-91d32edba874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:23.523065 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4943e-3f13-4c0a-8572-91d32edba874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:23.523330 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4943e-3f13-4c0a-8572-91d32edba874 admin admin] PATH_INFO: `/v3/projects/15af566aad214a768abb25bc7bcc94b9` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:23.525252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb4943e-3f13-4c0a-8572-91d32edba874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15af566aad214a768abb25bc7bcc94b9)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:23.541100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb4943e-3f13-4c0a-8572-91d32edba874 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:23.592375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7eb4943e-3f13-4c0a-8572-91d32edba874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15af566aad214a768abb25bc7bcc94b9', 'request_id': 'req-7eb4943e-3f13-4c0a-8572-91d32edba874'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:23.599278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:23.611598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7eb4943e-3f13-4c0a-8572-91d32edba874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15af566aad214a768abb25bc7bcc94b9', 'request_id': 'req-7eb4943e-3f13-4c0a-8572-91d32edba874'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:23.622462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7eb4943e-3f13-4c0a-8572-91d32edba874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15af566aad214a768abb25bc7bcc94b9', 'request_id': 'req-7eb4943e-3f13-4c0a-8572-91d32edba874'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:23.637746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7eb4943e-3f13-4c0a-8572-91d32edba874 admin admin] The token cache is being invalidate because project 15af566aad214a768abb25bc7bcc94b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:23.638448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7eb4943e-3f13-4c0a-8572-91d32edba874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:23.639027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7eb4943e-3f13-4c0a-8572-91d32edba874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:23.641767 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1075/2160] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:26:23 2022] DELETE /identity/v3/projects/15af566aad214a768abb25bc7bcc94b9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:23.647362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecfe87-3f4c-4197-a88c-a35273e54e6b None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:23.647680 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecfe87-3f4c-4197-a88c-a35273e54e6b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:23.648534 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecfe87-3f4c-4197-a88c-a35273e54e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:23.680123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-87ecfe87-3f4c-4197-a88c-a35273e54e6b None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:23.681662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-87ecfe87-3f4c-4197-a88c-a35273e54e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:23.747044 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1086/2161] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:23.756059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d4c09-bf9e-4e51-ada3-0cbf2a9f36b1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:23.756243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d4c09-bf9e-4e51-ada3-0cbf2a9f36b1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:23.756523 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d4c09-bf9e-4e51-ada3-0cbf2a9f36b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:23.777882 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-701d4c09-bf9e-4e51-ada3-0cbf2a9f36b1 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:23.779165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-701d4c09-bf9e-4e51-ada3-0cbf2a9f36b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:23.787496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1076/2162] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:23.792845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecfe87-3f4c-4197-a88c-a35273e54e6b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:23.804200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecfe87-3f4c-4197-a88c-a35273e54e6b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:23.810057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1826c-8fa9-458f-bd6b-c229fe0dabc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:23.810694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1826c-8fa9-458f-bd6b-c229fe0dabc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:23.811135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1826c-8fa9-458f-bd6b-c229fe0dabc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:23.811489 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1826c-8fa9-458f-bd6b-c229fe0dabc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:23.812124 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1826c-8fa9-458f-bd6b-c229fe0dabc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:26:23.812432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1826c-8fa9-458f-bd6b-c229fe0dabc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:23.814117 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1826c-8fa9-458f-bd6b-c229fe0dabc9 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:23.826906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1087/2163] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:26:23.831676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d4c09-bf9e-4e51-ada3-0cbf2a9f36b1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:23.841871 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d4c09-bf9e-4e51-ada3-0cbf2a9f36b1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:23.847576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3b900-d286-48ac-a8d4-94c01278cf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:23.848187 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3b900-d286-48ac-a8d4-94c01278cf57 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:23.848424 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3b900-d286-48ac-a8d4-94c01278cf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:23.848710 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3b900-d286-48ac-a8d4-94c01278cf57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:23.849381 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3b900-d286-48ac-a8d4-94c01278cf57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:23.851929 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3b900-d286-48ac-a8d4-94c01278cf57 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:23.887332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1077/2164] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:26:23 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:23.891626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1826c-8fa9-458f-bd6b-c229fe0dabc9 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:23.913770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1826c-8fa9-458f-bd6b-c229fe0dabc9 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:23.984000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13a399-2c47-4a0e-9d80-3abc4a11e172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:23.984693 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13a399-2c47-4a0e-9d80-3abc4a11e172 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:23.984973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13a399-2c47-4a0e-9d80-3abc4a11e172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:23.985482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13a399-2c47-4a0e-9d80-3abc4a11e172 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:23.985962 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13a399-2c47-4a0e-9d80-3abc4a11e172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:23.987894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13a399-2c47-4a0e-9d80-3abc4a11e172 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:23.997523 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:24.013198 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1088/2165] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:26:23 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:24.017273 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3b900-d286-48ac-a8d4-94c01278cf57 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:24.029934 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3b900-d286-48ac-a8d4-94c01278cf57 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:24.034663 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d0ebe-2654-4a37-bc17-90337bc8e549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:24.035291 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d0ebe-2654-4a37-bc17-90337bc8e549 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:24.035581 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d0ebe-2654-4a37-bc17-90337bc8e549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:24.035834 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d0ebe-2654-4a37-bc17-90337bc8e549 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:24.036679 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d0ebe-2654-4a37-bc17-90337bc8e549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:24.038526 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d0ebe-2654-4a37-bc17-90337bc8e549 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:24.051164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1078/2166] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:24.055508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13a399-2c47-4a0e-9d80-3abc4a11e172 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:24.069140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13a399-2c47-4a0e-9d80-3abc4a11e172 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:24.073514 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d126587-90b9-41a3-8b0f-c17772691ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:24.074676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d126587-90b9-41a3-8b0f-c17772691ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:24.074676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d126587-90b9-41a3-8b0f-c17772691ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:24.074911 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d126587-90b9-41a3-8b0f-c17772691ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:24.075485 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d126587-90b9-41a3-8b0f-c17772691ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:24.077138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d126587-90b9-41a3-8b0f-c17772691ff7 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:24.091767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1089/2167] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:24.096824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d0ebe-2654-4a37-bc17-90337bc8e549 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:24.107917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d0ebe-2654-4a37-bc17-90337bc8e549 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:24.114793 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43fe9e-4bcd-45b9-9664-584caa2036ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:24.117841 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43fe9e-4bcd-45b9-9664-584caa2036ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:24.117841 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43fe9e-4bcd-45b9-9664-584caa2036ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:24.117841 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43fe9e-4bcd-45b9-9664-584caa2036ca admin admin] PATH_INFO: `/v3/projects/d2e3c8ce1920456886466e387203b567/users/04429689eb9a4c2182c46cf43a3a5c55/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:24.133095 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43fe9e-4bcd-45b9-9664-584caa2036ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2e3c8ce1920456886466e387203b567, user_id=04429689eb9a4c2182c46cf43a3a5c55, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:24.135922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43fe9e-4bcd-45b9-9664-584caa2036ca admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:24.150745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1079/2168] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:24 2022] PUT /identity/v3/projects/d2e3c8ce1920456886466e387203b567/users/04429689eb9a4c2182c46cf43a3a5c55/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:24.161171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9be83-83f2-404a-ac7d-2a39cd76bfb0 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:24.161436 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9be83-83f2-404a-ac7d-2a39cd76bfb0 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:24.161710 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9be83-83f2-404a-ac7d-2a39cd76bfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:24.191662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-ead9be83-83f2-404a-ac7d-2a39cd76bfb0 None None] MFA Rules not processed for user `04429689eb9a4c2182c46cf43a3a5c55`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:24.192893 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-ead9be83-83f2-404a-ac7d-2a39cd76bfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:24.292139 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1090/2169] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:24.296728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43fe9e-4bcd-45b9-9664-584caa2036ca admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:24.366120 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43fe9e-4bcd-45b9-9664-584caa2036ca admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:24.371675 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627c4fe-a05f-4f9b-a499-aedc1febe0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:24.372317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627c4fe-a05f-4f9b-a499-aedc1febe0b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:24.372596 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627c4fe-a05f-4f9b-a499-aedc1febe0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:24.372873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627c4fe-a05f-4f9b-a499-aedc1febe0b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:24.377529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627c4fe-a05f-4f9b-a499-aedc1febe0b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:24.377529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627c4fe-a05f-4f9b-a499-aedc1febe0b3 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:24.415601 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1080/2170] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:26:24 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:24.419961 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9be83-83f2-404a-ac7d-2a39cd76bfb0 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:24.431319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9be83-83f2-404a-ac7d-2a39cd76bfb0 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:24.493773 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06e052-d01c-4acc-921c-479c624c9eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:24.494732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06e052-d01c-4acc-921c-479c624c9eaf admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:24.495109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06e052-d01c-4acc-921c-479c624c9eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:24.495454 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06e052-d01c-4acc-921c-479c624c9eaf admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:24.496397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06e052-d01c-4acc-921c-479c624c9eaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:24.499194 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06e052-d01c-4acc-921c-479c624c9eaf admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:24.528322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1091/2171] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:26:24 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:26:24.529911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627c4fe-a05f-4f9b-a499-aedc1febe0b3 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:24.541701 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627c4fe-a05f-4f9b-a499-aedc1febe0b3 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:24.552201 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc4ba5-7307-4899-97da-ec6e3dc37334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:24.552926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc4ba5-7307-4899-97da-ec6e3dc37334 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:24.553206 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc4ba5-7307-4899-97da-ec6e3dc37334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:24.553478 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc4ba5-7307-4899-97da-ec6e3dc37334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:24.554389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc4ba5-7307-4899-97da-ec6e3dc37334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:24.556191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc4ba5-7307-4899-97da-ec6e3dc37334 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:24.579232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1081/2172] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:24.583455 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06e052-d01c-4acc-921c-479c624c9eaf admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:24.602850 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06e052-d01c-4acc-921c-479c624c9eaf admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:24.609996 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019d12a-aede-44cd-a3fe-854ac3994ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:24.610718 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019d12a-aede-44cd-a3fe-854ac3994ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:24.611105 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019d12a-aede-44cd-a3fe-854ac3994ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:24.611477 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019d12a-aede-44cd-a3fe-854ac3994ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:24.612324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019d12a-aede-44cd-a3fe-854ac3994ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:24.614293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019d12a-aede-44cd-a3fe-854ac3994ba3 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:24.630901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1092/2173] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:24.633137 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc4ba5-7307-4899-97da-ec6e3dc37334 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:24.664293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc4ba5-7307-4899-97da-ec6e3dc37334 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:24.671242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2d599-0432-4a89-a0dd-100e49ea1566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:24.672078 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2d599-0432-4a89-a0dd-100e49ea1566 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:24.672458 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2d599-0432-4a89-a0dd-100e49ea1566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:24.672848 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2d599-0432-4a89-a0dd-100e49ea1566 admin admin] PATH_INFO: `/v3/projects/f714353fa47046be97581b6f9aab943b/users/368dbc538517425a8f68d5def554f6ea/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:24.695146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2d599-0432-4a89-a0dd-100e49ea1566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f714353fa47046be97581b6f9aab943b, user_id=368dbc538517425a8f68d5def554f6ea, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:24.699965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2d599-0432-4a89-a0dd-100e49ea1566 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:24.733331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1082/2174] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:24 2022] PUT /identity/v3/projects/f714353fa47046be97581b6f9aab943b/users/368dbc538517425a8f68d5def554f6ea/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:24.737447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76798624-3ffe-4a05-909a-59eb19c32301 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:24.738804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76798624-3ffe-4a05-909a-59eb19c32301 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:24.739090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76798624-3ffe-4a05-909a-59eb19c32301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:24.790933 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-76798624-3ffe-4a05-909a-59eb19c32301 None None] MFA Rules not processed for user `368dbc538517425a8f68d5def554f6ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:24.793366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-76798624-3ffe-4a05-909a-59eb19c32301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:24.898454 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:24.955688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1093/2175] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:24.960601 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2d599-0432-4a89-a0dd-100e49ea1566 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:24.998225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:25.043292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2d599-0432-4a89-a0dd-100e49ea1566 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:25.049812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76a87c-b7f7-49bc-b389-80115fd6a65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:25.050458 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76a87c-b7f7-49bc-b389-80115fd6a65c admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:25.050735 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76a87c-b7f7-49bc-b389-80115fd6a65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:25.050999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76a87c-b7f7-49bc-b389-80115fd6a65c admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:25.051668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76a87c-b7f7-49bc-b389-80115fd6a65c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:25.053592 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba76a87c-b7f7-49bc-b389-80115fd6a65c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:25.085668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1083/2176] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:26:24 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:25.091531 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76798624-3ffe-4a05-909a-59eb19c32301 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:25.113598 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76798624-3ffe-4a05-909a-59eb19c32301 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:25.183633 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b37bf6-eee7-4d75-82a0-ec55c841a88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:25.184378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b37bf6-eee7-4d75-82a0-ec55c841a88b admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:25.184718 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b37bf6-eee7-4d75-82a0-ec55c841a88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:25.184996 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b37bf6-eee7-4d75-82a0-ec55c841a88b admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:25.185663 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b37bf6-eee7-4d75-82a0-ec55c841a88b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:25.187926 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b37bf6-eee7-4d75-82a0-ec55c841a88b admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:25.217857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1094/2177] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:26:25 2022] POST /identity/v3/users => generated 298 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:25.219577 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76a87c-b7f7-49bc-b389-80115fd6a65c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:25.230421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba76a87c-b7f7-49bc-b389-80115fd6a65c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:25.235421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe08ea4-11a3-4c5e-a86f-c3272d539f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:25.236082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe08ea4-11a3-4c5e-a86f-c3272d539f5f admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:25.236371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe08ea4-11a3-4c5e-a86f-c3272d539f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:25.236643 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe08ea4-11a3-4c5e-a86f-c3272d539f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:25.237357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe08ea4-11a3-4c5e-a86f-c3272d539f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:25.239051 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe08ea4-11a3-4c5e-a86f-c3272d539f5f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:25.251924 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1084/2178] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:26:25.256454 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b37bf6-eee7-4d75-82a0-ec55c841a88b admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:25.268161 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b37bf6-eee7-4d75-82a0-ec55c841a88b admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:25.274436 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bdede0-d219-410b-8f85-993811cc3385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:25.275130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdede0-d219-410b-8f85-993811cc3385 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:25.275414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdede0-d219-410b-8f85-993811cc3385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:25.275690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bdede0-d219-410b-8f85-993811cc3385 admin admin] PATH_INFO: `/v3/projects/14e930fc041640d6afce6867c99233fd/users/770b0fcaddbc4bca9d2dee4b0e941a56/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:25.295094 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bdede0-d219-410b-8f85-993811cc3385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14e930fc041640d6afce6867c99233fd, user_id=770b0fcaddbc4bca9d2dee4b0e941a56, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:25.297657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bdede0-d219-410b-8f85-993811cc3385 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:25.314192 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1095/2179] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:25 2022] PUT /identity/v3/projects/14e930fc041640d6afce6867c99233fd/users/770b0fcaddbc4bca9d2dee4b0e941a56/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:26:25.318501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe08ea4-11a3-4c5e-a86f-c3272d539f5f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:25.329274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe08ea4-11a3-4c5e-a86f-c3272d539f5f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:25.391723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0bb0e0-6414-4d0b-88c1-b0a89a582c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:25.392466 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0bb0e0-6414-4d0b-88c1-b0a89a582c47 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:25.392889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0bb0e0-6414-4d0b-88c1-b0a89a582c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:25.393235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0bb0e0-6414-4d0b-88c1-b0a89a582c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:25.394287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0bb0e0-6414-4d0b-88c1-b0a89a582c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:25.396746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0bb0e0-6414-4d0b-88c1-b0a89a582c47 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:25.409460 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1085/2180] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:25.414358 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bdede0-d219-410b-8f85-993811cc3385 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:25.425859 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bdede0-d219-410b-8f85-993811cc3385 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:25.432267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f903565-cb74-428a-b745-e962bbd100d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:25.432973 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f903565-cb74-428a-b745-e962bbd100d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:25.433240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f903565-cb74-428a-b745-e962bbd100d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:25.433519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f903565-cb74-428a-b745-e962bbd100d2 admin admin] PATH_INFO: `/v3/projects/14e930fc041640d6afce6867c99233fd/users/770b0fcaddbc4bca9d2dee4b0e941a56/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:25.436043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f903565-cb74-428a-b745-e962bbd100d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14e930fc041640d6afce6867c99233fd, user_id=770b0fcaddbc4bca9d2dee4b0e941a56, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:25.438816 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f903565-cb74-428a-b745-e962bbd100d2 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:25.451505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1096/2181] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:25 2022] PUT /identity/v3/projects/14e930fc041640d6afce6867c99233fd/users/770b0fcaddbc4bca9d2dee4b0e941a56/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:25.463781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a625b1-d50a-487f-9f61-40c29b87e51b None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:25.464063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a625b1-d50a-487f-9f61-40c29b87e51b None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:25.464347 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a625b1-d50a-487f-9f61-40c29b87e51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:25.496899 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-23a625b1-d50a-487f-9f61-40c29b87e51b None None] MFA Rules not processed for user `770b0fcaddbc4bca9d2dee4b0e941a56`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:25.498275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-23a625b1-d50a-487f-9f61-40c29b87e51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:25.580142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1086/2182] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:25.595629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f903565-cb74-428a-b745-e962bbd100d2 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:25.597678 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9f903565-cb74-428a-b745-e962bbd100d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:25.660424 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f903565-cb74-428a-b745-e962bbd100d2 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:25.665224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e31b8-90e9-4f31-bad7-27a196e4ecc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:25.665876 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e31b8-90e9-4f31-bad7-27a196e4ecc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:25.666150 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e31b8-90e9-4f31-bad7-27a196e4ecc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:25.666550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e31b8-90e9-4f31-bad7-27a196e4ecc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:25.716953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:25.733846 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e31b8-90e9-4f31-bad7-27a196e4ecc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:25.735350 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e31b8-90e9-4f31-bad7-27a196e4ecc7 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:25.745628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1097/2183] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:27.343433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a625b1-d50a-487f-9f61-40c29b87e51b None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:27.352191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a625b1-d50a-487f-9f61-40c29b87e51b None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:27.356203 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5575736-a03e-4bdb-aaf2-51539e1cd5a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:27.356930 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5575736-a03e-4bdb-aaf2-51539e1cd5a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:27.357108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5575736-a03e-4bdb-aaf2-51539e1cd5a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:27.357387 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5575736-a03e-4bdb-aaf2-51539e1cd5a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:27.383940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:27.456434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5575736-a03e-4bdb-aaf2-51539e1cd5a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:27.458429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5575736-a03e-4bdb-aaf2-51539e1cd5a9 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:27.470773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1087/2184] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:27.718300 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe0a39-f81d-4653-bff3-63a92f1be265 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:27.718995 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe0a39-f81d-4653-bff3-63a92f1be265 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:27.718995 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe0a39-f81d-4653-bff3-63a92f1be265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:27.754207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-b7fe0a39-f81d-4653-bff3-63a92f1be265 None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:27.755630 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-b7fe0a39-f81d-4653-bff3-63a92f1be265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:27.825136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1098/2185] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 15:26:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:27.839847 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5575736-a03e-4bdb-aaf2-51539e1cd5a9 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:27.841761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d5575736-a03e-4bdb-aaf2-51539e1cd5a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:27.944527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5575736-a03e-4bdb-aaf2-51539e1cd5a9 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:27.955574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb74d21-d26a-4249-a4ef-c692f8731baf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:27.956823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb74d21-d26a-4249-a4ef-c692f8731baf service swift] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:27.957297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb74d21-d26a-4249-a4ef-c692f8731baf service swift] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:27.957621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb74d21-d26a-4249-a4ef-c692f8731baf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:27.971495 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb74d21-d26a-4249-a4ef-c692f8731baf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:27.973232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb74d21-d26a-4249-a4ef-c692f8731baf service swift] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:27.994527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1088/2186] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:26:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:26:28.244667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe0a39-f81d-4653-bff3-63a92f1be265 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:28.258430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe0a39-f81d-4653-bff3-63a92f1be265 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:28.262983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846dc16-c72b-4a69-9e35-45121f8dda61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:28.263620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846dc16-c72b-4a69-9e35-45121f8dda61 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:28.263915 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846dc16-c72b-4a69-9e35-45121f8dda61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:28.264168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846dc16-c72b-4a69-9e35-45121f8dda61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:28.276639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846dc16-c72b-4a69-9e35-45121f8dda61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:28.279682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846dc16-c72b-4a69-9e35-45121f8dda61 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:28.295723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1099/2187] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:32.589744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb74d21-d26a-4249-a4ef-c692f8731baf service swift] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:32.676607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb74d21-d26a-4249-a4ef-c692f8731baf service swift] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:32.681405 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e616786-e8fa-4efd-9d54-b79210710633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:32.682247 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e616786-e8fa-4efd-9d54-b79210710633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:32.682620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e616786-e8fa-4efd-9d54-b79210710633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:32.682987 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e616786-e8fa-4efd-9d54-b79210710633 admin admin] PATH_INFO: `/v3/users/04429689eb9a4c2182c46cf43a3a5c55` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:32.684264 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e616786-e8fa-4efd-9d54-b79210710633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04429689eb9a4c2182c46cf43a3a5c55)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:32.686329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e616786-e8fa-4efd-9d54-b79210710633 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:32.743715 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2e616786-e8fa-4efd-9d54-b79210710633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04429689eb9a4c2182c46cf43a3a5c55'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:32.744548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2e616786-e8fa-4efd-9d54-b79210710633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04429689eb9a4c2182c46cf43a3a5c55'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:32.749761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:32.760094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2e616786-e8fa-4efd-9d54-b79210710633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04429689eb9a4c2182c46cf43a3a5c55'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:32.769096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2e616786-e8fa-4efd-9d54-b79210710633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04429689eb9a4c2182c46cf43a3a5c55'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:32.778329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2e616786-e8fa-4efd-9d54-b79210710633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04429689eb9a4c2182c46cf43a3a5c55'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:32.789324 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1089/2188] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:26:32 2022] DELETE /identity/v3/users/04429689eb9a4c2182c46cf43a3a5c55 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:26:32.794140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7ebe5-c2f8-4ab4-9cc4-b0b0e6335b16 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:32.795177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7ebe5-c2f8-4ab4-9cc4-b0b0e6335b16 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:32.795614 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7ebe5-c2f8-4ab4-9cc4-b0b0e6335b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:32.817597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-95e7ebe5-c2f8-4ab4-9cc4-b0b0e6335b16 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:32.820157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-95e7ebe5-c2f8-4ab4-9cc4-b0b0e6335b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:32.930577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1100/2189] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:32.941809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e616786-e8fa-4efd-9d54-b79210710633 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:32.943326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2e616786-e8fa-4efd-9d54-b79210710633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:33.023462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e616786-e8fa-4efd-9d54-b79210710633 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:33.029365 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c12e6-2cfc-4335-bcb7-be7526150a8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:33.030767 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c12e6-2cfc-4335-bcb7-be7526150a8b service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:33.030767 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c12e6-2cfc-4335-bcb7-be7526150a8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:33.031046 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c12e6-2cfc-4335-bcb7-be7526150a8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:33.042504 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c12e6-2cfc-4335-bcb7-be7526150a8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:33.044448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c12e6-2cfc-4335-bcb7-be7526150a8b service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:33.068359 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1090/2190] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:33.554046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7ebe5-c2f8-4ab4-9cc4-b0b0e6335b16 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:33.554046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-95e7ebe5-c2f8-4ab4-9cc4-b0b0e6335b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:33.578338 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7ebe5-c2f8-4ab4-9cc4-b0b0e6335b16 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:33.602069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78584e6-c966-49d6-8b37-4330dbd81b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:33.603311 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78584e6-c966-49d6-8b37-4330dbd81b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:33.603886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78584e6-c966-49d6-8b37-4330dbd81b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:33.604347 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78584e6-c966-49d6-8b37-4330dbd81b79 admin admin] PATH_INFO: `/v3/projects/d2e3c8ce1920456886466e387203b567` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:33.606319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78584e6-c966-49d6-8b37-4330dbd81b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2e3c8ce1920456886466e387203b567)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:33.609295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78584e6-c966-49d6-8b37-4330dbd81b79 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:33.665129 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c78584e6-c966-49d6-8b37-4330dbd81b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2e3c8ce1920456886466e387203b567', 'request_id': 'req-c78584e6-c966-49d6-8b37-4330dbd81b79'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:33.676977 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c78584e6-c966-49d6-8b37-4330dbd81b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2e3c8ce1920456886466e387203b567', 'request_id': 'req-c78584e6-c966-49d6-8b37-4330dbd81b79'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:33.685752 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c78584e6-c966-49d6-8b37-4330dbd81b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2e3c8ce1920456886466e387203b567', 'request_id': 'req-c78584e6-c966-49d6-8b37-4330dbd81b79'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:33.700803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:33.708896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c78584e6-c966-49d6-8b37-4330dbd81b79 admin admin] The token cache is being invalidate because project d2e3c8ce1920456886466e387203b567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:33.709825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c78584e6-c966-49d6-8b37-4330dbd81b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:33.710663 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c78584e6-c966-49d6-8b37-4330dbd81b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:33.713121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1101/2191] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:26:33 2022] DELETE /identity/v3/projects/d2e3c8ce1920456886466e387203b567 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:33.717383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c12e6-2cfc-4335-bcb7-be7526150a8b service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:33.728018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c12e6-2cfc-4335-bcb7-be7526150a8b service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:33.825045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437f9b2-8993-4377-90a3-da57a64008c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:33.826084 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437f9b2-8993-4377-90a3-da57a64008c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:33.826084 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437f9b2-8993-4377-90a3-da57a64008c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:33.826309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437f9b2-8993-4377-90a3-da57a64008c5 admin admin] PATH_INFO: `/v3/users/368dbc538517425a8f68d5def554f6ea` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:33.827564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437f9b2-8993-4377-90a3-da57a64008c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=368dbc538517425a8f68d5def554f6ea)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:33.829582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437f9b2-8993-4377-90a3-da57a64008c5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:33.895419 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7437f9b2-8993-4377-90a3-da57a64008c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '368dbc538517425a8f68d5def554f6ea'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:33.896370 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7437f9b2-8993-4377-90a3-da57a64008c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '368dbc538517425a8f68d5def554f6ea'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:33.909300 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7437f9b2-8993-4377-90a3-da57a64008c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '368dbc538517425a8f68d5def554f6ea'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:33.920022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7437f9b2-8993-4377-90a3-da57a64008c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '368dbc538517425a8f68d5def554f6ea'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:33.923698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:33.936389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7437f9b2-8993-4377-90a3-da57a64008c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '368dbc538517425a8f68d5def554f6ea'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:33.952306 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1091/2192] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:26:33 2022] DELETE /identity/v3/users/368dbc538517425a8f68d5def554f6ea => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:34.455878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78584e6-c966-49d6-8b37-4330dbd81b79 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:34.458936 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c78584e6-c966-49d6-8b37-4330dbd81b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:34.493467 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78584e6-c966-49d6-8b37-4330dbd81b79 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:34.580080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d44e9c-d163-409e-8010-b86e7cb9a83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:34.581423 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d44e9c-d163-409e-8010-b86e7cb9a83d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:34.581957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d44e9c-d163-409e-8010-b86e7cb9a83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:34.582445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d44e9c-d163-409e-8010-b86e7cb9a83d admin admin] PATH_INFO: `/v3/projects/f714353fa47046be97581b6f9aab943b` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:34.583932 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d44e9c-d163-409e-8010-b86e7cb9a83d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f714353fa47046be97581b6f9aab943b)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:34.586583 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d44e9c-d163-409e-8010-b86e7cb9a83d admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:34.661409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-37d44e9c-d163-409e-8010-b86e7cb9a83d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f714353fa47046be97581b6f9aab943b', 'request_id': 'req-37d44e9c-d163-409e-8010-b86e7cb9a83d'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:34.673328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-37d44e9c-d163-409e-8010-b86e7cb9a83d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f714353fa47046be97581b6f9aab943b', 'request_id': 'req-37d44e9c-d163-409e-8010-b86e7cb9a83d'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:34.680788 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-37d44e9c-d163-409e-8010-b86e7cb9a83d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f714353fa47046be97581b6f9aab943b', 'request_id': 'req-37d44e9c-d163-409e-8010-b86e7cb9a83d'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:34.693241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-37d44e9c-d163-409e-8010-b86e7cb9a83d admin admin] The token cache is being invalidate because project f714353fa47046be97581b6f9aab943b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:34.693885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-37d44e9c-d163-409e-8010-b86e7cb9a83d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:34.694636 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-37d44e9c-d163-409e-8010-b86e7cb9a83d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:34.696547 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1102/2193] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:26:34 2022] DELETE /identity/v3/projects/f714353fa47046be97581b6f9aab943b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:34.700570 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437f9b2-8993-4377-90a3-da57a64008c5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:34.702097 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7437f9b2-8993-4377-90a3-da57a64008c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:34.725157 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437f9b2-8993-4377-90a3-da57a64008c5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:34.790346 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15494d-b2b6-4ed3-b9f4-c631d2f6d515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:34.791078 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15494d-b2b6-4ed3-b9f4-c631d2f6d515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:34.791358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15494d-b2b6-4ed3-b9f4-c631d2f6d515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:34.791638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15494d-b2b6-4ed3-b9f4-c631d2f6d515 admin admin] PATH_INFO: `/v3/users/770b0fcaddbc4bca9d2dee4b0e941a56` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:34.793074 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15494d-b2b6-4ed3-b9f4-c631d2f6d515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=770b0fcaddbc4bca9d2dee4b0e941a56)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:34.796279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15494d-b2b6-4ed3-b9f4-c631d2f6d515 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:34.887697 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5a15494d-b2b6-4ed3-b9f4-c631d2f6d515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770b0fcaddbc4bca9d2dee4b0e941a56'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:34.888463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5a15494d-b2b6-4ed3-b9f4-c631d2f6d515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770b0fcaddbc4bca9d2dee4b0e941a56'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:34.904643 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5a15494d-b2b6-4ed3-b9f4-c631d2f6d515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770b0fcaddbc4bca9d2dee4b0e941a56'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:34.918107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5a15494d-b2b6-4ed3-b9f4-c631d2f6d515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770b0fcaddbc4bca9d2dee4b0e941a56'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:34.927501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5a15494d-b2b6-4ed3-b9f4-c631d2f6d515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770b0fcaddbc4bca9d2dee4b0e941a56'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:34.935699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1092/2194] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:26:34 2022] DELETE /identity/v3/users/770b0fcaddbc4bca9d2dee4b0e941a56 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:35.518589 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d44e9c-d163-409e-8010-b86e7cb9a83d admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:35.520581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-37d44e9c-d163-409e-8010-b86e7cb9a83d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:35.550416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d44e9c-d163-409e-8010-b86e7cb9a83d admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:35.746573 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae0cad-d1be-4973-91b1-970085ef2ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:35.747271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae0cad-d1be-4973-91b1-970085ef2ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:35.747554 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae0cad-d1be-4973-91b1-970085ef2ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:35.747823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae0cad-d1be-4973-91b1-970085ef2ae0 admin admin] PATH_INFO: `/v3/projects/14e930fc041640d6afce6867c99233fd` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:35.749356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae0cad-d1be-4973-91b1-970085ef2ae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14e930fc041640d6afce6867c99233fd)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:35.751349 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae0cad-d1be-4973-91b1-970085ef2ae0 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:35.833018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:35.884934 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-95ae0cad-d1be-4973-91b1-970085ef2ae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14e930fc041640d6afce6867c99233fd', 'request_id': 'req-95ae0cad-d1be-4973-91b1-970085ef2ae0'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.897185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-95ae0cad-d1be-4973-91b1-970085ef2ae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14e930fc041640d6afce6867c99233fd', 'request_id': 'req-95ae0cad-d1be-4973-91b1-970085ef2ae0'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.916598 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-95ae0cad-d1be-4973-91b1-970085ef2ae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14e930fc041640d6afce6867c99233fd', 'request_id': 'req-95ae0cad-d1be-4973-91b1-970085ef2ae0'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.937683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-95ae0cad-d1be-4973-91b1-970085ef2ae0 admin admin] The token cache is being invalidate because project 14e930fc041640d6afce6867c99233fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:35.938244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-95ae0cad-d1be-4973-91b1-970085ef2ae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.943815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-95ae0cad-d1be-4973-91b1-970085ef2ae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:35.957815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1103/2195] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:26:35 2022] DELETE /identity/v3/projects/14e930fc041640d6afce6867c99233fd => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:35.963475 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3779a4-666f-463f-85b9-0e5a1bfab5c7 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:35.963475 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3779a4-666f-463f-85b9-0e5a1bfab5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:35.963475 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3779a4-666f-463f-85b9-0e5a1bfab5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:35.984518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-bf3779a4-666f-463f-85b9-0e5a1bfab5c7 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:35.985982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bf3779a4-666f-463f-85b9-0e5a1bfab5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:36.063793 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1093/2196] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:36.071590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9d432-f257-494c-8385-c24dc82dca2f None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.072175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9d432-f257-494c-8385-c24dc82dca2f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.072305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9d432-f257-494c-8385-c24dc82dca2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.113058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2bc9d432-f257-494c-8385-c24dc82dca2f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:36.115150 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2bc9d432-f257-494c-8385-c24dc82dca2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:36.129676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1104/2197] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:36.131607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3779a4-666f-463f-85b9-0e5a1bfab5c7 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:36.143998 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3779a4-666f-463f-85b9-0e5a1bfab5c7 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:36.157160 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0af87-510f-4b2e-9103-6d9b54d14540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:36.158283 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0af87-510f-4b2e-9103-6d9b54d14540 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.162234 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0af87-510f-4b2e-9103-6d9b54d14540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.162706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0af87-510f-4b2e-9103-6d9b54d14540 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.163679 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0af87-510f-4b2e-9103-6d9b54d14540 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:26:36.164229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0af87-510f-4b2e-9103-6d9b54d14540 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:36.166398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0af87-510f-4b2e-9103-6d9b54d14540 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:36.176124 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:36.190829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1094/2198] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:26:36.191571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9d432-f257-494c-8385-c24dc82dca2f None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:36.205483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9d432-f257-494c-8385-c24dc82dca2f None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:36.213975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895e06a-62e2-4169-a1f1-4e89ec81b99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:36.214941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895e06a-62e2-4169-a1f1-4e89ec81b99a admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.215822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895e06a-62e2-4169-a1f1-4e89ec81b99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.215822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895e06a-62e2-4169-a1f1-4e89ec81b99a admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.216653 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895e06a-62e2-4169-a1f1-4e89ec81b99a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:36.221318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895e06a-62e2-4169-a1f1-4e89ec81b99a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:36.278782 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1105/2199] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:26:36 2022] POST /identity/v3/projects => generated 389 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:26:36.283714 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0af87-510f-4b2e-9103-6d9b54d14540 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:36.297397 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0af87-510f-4b2e-9103-6d9b54d14540 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:36.422160 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03038e-04fb-43b0-b40d-a510dd3a0976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:36.422811 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03038e-04fb-43b0-b40d-a510dd3a0976 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.423086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03038e-04fb-43b0-b40d-a510dd3a0976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.423358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03038e-04fb-43b0-b40d-a510dd3a0976 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.424030 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03038e-04fb-43b0-b40d-a510dd3a0976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:36.426015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03038e-04fb-43b0-b40d-a510dd3a0976 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:36.465079 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1095/2200] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:26:36 2022] POST /identity/v3/users => generated 295 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:36.466401 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895e06a-62e2-4169-a1f1-4e89ec81b99a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:36.488247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895e06a-62e2-4169-a1f1-4e89ec81b99a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:36.493807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb7a1b-dfe5-4ef1-bb61-0dad56381664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:36.494824 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb7a1b-dfe5-4ef1-bb61-0dad56381664 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.495127 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb7a1b-dfe5-4ef1-bb61-0dad56381664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.495401 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb7a1b-dfe5-4ef1-bb61-0dad56381664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.496306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb7a1b-dfe5-4ef1-bb61-0dad56381664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:36.498356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb7a1b-dfe5-4ef1-bb61-0dad56381664 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:36.524794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03038e-04fb-43b0-b40d-a510dd3a0976 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:36.526130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1106/2201] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:36.540505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03038e-04fb-43b0-b40d-a510dd3a0976 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:36.552187 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a840c-edd1-4700-9b6b-b65727d9abd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:36.553334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a840c-edd1-4700-9b6b-b65727d9abd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.553764 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a840c-edd1-4700-9b6b-b65727d9abd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.554199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a840c-edd1-4700-9b6b-b65727d9abd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.555118 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a840c-edd1-4700-9b6b-b65727d9abd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:36.557472 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a840c-edd1-4700-9b6b-b65727d9abd4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:36.576479 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1096/2202] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:26:36.579564 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb7a1b-dfe5-4ef1-bb61-0dad56381664 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:36.591395 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb7a1b-dfe5-4ef1-bb61-0dad56381664 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:36.596630 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672915c4-2abe-4cd1-a687-4a28622a8edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:36.597425 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672915c4-2abe-4cd1-a687-4a28622a8edf admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.597855 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672915c4-2abe-4cd1-a687-4a28622a8edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.598162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672915c4-2abe-4cd1-a687-4a28622a8edf admin admin] PATH_INFO: `/v3/projects/f4e32ad093cf41e082bcedd5072092d3/users/3ac66f9b870c44e287b21d6cc5e3f139/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.616178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672915c4-2abe-4cd1-a687-4a28622a8edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e32ad093cf41e082bcedd5072092d3, user_id=3ac66f9b870c44e287b21d6cc5e3f139, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:36.620213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672915c4-2abe-4cd1-a687-4a28622a8edf admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:36.648438 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1107/2203] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:36 2022] PUT /identity/v3/projects/f4e32ad093cf41e082bcedd5072092d3/users/3ac66f9b870c44e287b21d6cc5e3f139/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:36.660108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc74b2-bd1c-491c-b09b-c6af8f09d690 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.660485 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc74b2-bd1c-491c-b09b-c6af8f09d690 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.660854 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc74b2-bd1c-491c-b09b-c6af8f09d690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.693961 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-95bc74b2-bd1c-491c-b09b-c6af8f09d690 None None] MFA Rules not processed for user `3ac66f9b870c44e287b21d6cc5e3f139`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:36.695523 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-95bc74b2-bd1c-491c-b09b-c6af8f09d690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:36.790622 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1097/2204] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:36.802159 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672915c4-2abe-4cd1-a687-4a28622a8edf admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:36.804855 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-672915c4-2abe-4cd1-a687-4a28622a8edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:36.942522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672915c4-2abe-4cd1-a687-4a28622a8edf admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:36.953218 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bde5da-44af-4eeb-b102-49e756019bac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:36.954402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bde5da-44af-4eeb-b102-49e756019bac service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:36.955948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bde5da-44af-4eeb-b102-49e756019bac service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:36.957012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bde5da-44af-4eeb-b102-49e756019bac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:36.987775 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bde5da-44af-4eeb-b102-49e756019bac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:36.989725 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bde5da-44af-4eeb-b102-49e756019bac service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:37.023300 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1108/2205] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:46.777312 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=67,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:47.007079 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:49.869715 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc74b2-bd1c-491c-b09b-c6af8f09d690 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:49.871332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-95bc74b2-bd1c-491c-b09b-c6af8f09d690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:49.948252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc74b2-bd1c-491c-b09b-c6af8f09d690 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:49.953468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f86383-c9a9-4923-9fc5-bd77a2c2616b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '098d8d7e35804ccc931ddfba07f688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:49.954209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f86383-c9a9-4923-9fc5-bd77a2c2616b service placement] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:49.954491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f86383-c9a9-4923-9fc5-bd77a2c2616b service placement] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:49.954776 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f86383-c9a9-4923-9fc5-bd77a2c2616b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:49.957098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-40f86383-c9a9-4923-9fc5-bd77a2c2616b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:49.971134 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f86383-c9a9-4923-9fc5-bd77a2c2616b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:49.972844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f86383-c9a9-4923-9fc5-bd77a2c2616b service placement] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:49.984440 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1098/2206] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:51.386244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3e991-d99c-4231-adcf-c0a7dc084cf0 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:51.386799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3e991-d99c-4231-adcf-c0a7dc084cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:51.386799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3e991-d99c-4231-adcf-c0a7dc084cf0 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:51.388305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1109/2207] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 15:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:51.394701 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584a41a-7578-488a-bc6a-edebed530bea None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:51.394701 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584a41a-7578-488a-bc6a-edebed530bea None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:51.395672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584a41a-7578-488a-bc6a-edebed530bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:51.416639 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5584a41a-7578-488a-bc6a-edebed530bea None None] MFA Rules not processed for user `2de0ff99a2b74979a28a8b10e9ebdfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:51.416639 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5584a41a-7578-488a-bc6a-edebed530bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:51.491404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1099/2208] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 15:26:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:51.502618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3e991-d99c-4231-adcf-c0a7dc084cf0 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:51.504471 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9ae3e991-d99c-4231-adcf-c0a7dc084cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:51.565008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3e991-d99c-4231-adcf-c0a7dc084cf0 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:51.568452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17251c-e7dc-4a0f-b300-860dc0c01c31 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f696dfe4b34c4aa19104b5f7ee6e70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:51.569101 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17251c-e7dc-4a0f-b300-860dc0c01c31 service glance] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:51.569370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17251c-e7dc-4a0f-b300-860dc0c01c31 service glance] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:51.569618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17251c-e7dc-4a0f-b300-860dc0c01c31 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:51.581527 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17251c-e7dc-4a0f-b300-860dc0c01c31 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:51.583025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17251c-e7dc-4a0f-b300-860dc0c01c31 service glance] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:51.594430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1110/2209] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:51.671732 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f2b9c-92af-4ff7-af84-3001be9a1ea2 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:51.672020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f2b9c-92af-4ff7-af84-3001be9a1ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:51.672332 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f2b9c-92af-4ff7-af84-3001be9a1ea2 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:51.673749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1100/2210] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 15:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 15:26:51.678716 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193290a-1803-4074-a233-05e291c3a258 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:51.679010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193290a-1803-4074-a233-05e291c3a258 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:51.679310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193290a-1803-4074-a233-05e291c3a258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:51.703218 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-d193290a-1803-4074-a233-05e291c3a258 None None] MFA Rules not processed for user `5ef6577b10f7426a925d594e627f27cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:51.704466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d193290a-1803-4074-a233-05e291c3a258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:51.769077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1111/2211] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 15:26:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:51.783244 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f2b9c-92af-4ff7-af84-3001be9a1ea2 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:51.785193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-213f2b9c-92af-4ff7-af84-3001be9a1ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:51.800152 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f2b9c-92af-4ff7-af84-3001be9a1ea2 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:51.804152 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f41bf-59c4-481c-bec3-defb41e8ffef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:51.804778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f41bf-59c4-481c-bec3-defb41e8ffef service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:51.805057 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f41bf-59c4-481c-bec3-defb41e8ffef service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:51.805325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f41bf-59c4-481c-bec3-defb41e8ffef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:51.813083 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f41bf-59c4-481c-bec3-defb41e8ffef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:51.814689 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f41bf-59c4-481c-bec3-defb41e8ffef service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:51.823139 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1101/2212] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:51.909139 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165bc0c-1f0d-41a2-862b-e54c0665264c None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:51.909445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165bc0c-1f0d-41a2-862b-e54c0665264c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:51.909696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165bc0c-1f0d-41a2-862b-e54c0665264c None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:51.911092 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1112/2213] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 15:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:26:51.918257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a44eb-1dd7-4465-9944-8b2bcb71f3a1 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:51.918537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a44eb-1dd7-4465-9944-8b2bcb71f3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:51.918816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a44eb-1dd7-4465-9944-8b2bcb71f3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:51.939945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7d2a44eb-1dd7-4465-9944-8b2bcb71f3a1 None None] MFA Rules not processed for user `2de0ff99a2b74979a28a8b10e9ebdfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:51.941173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7d2a44eb-1dd7-4465-9944-8b2bcb71f3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:51.951324 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1102/2214] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 15:26:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:51.963285 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165bc0c-1f0d-41a2-862b-e54c0665264c None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:51.965417 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d165bc0c-1f0d-41a2-862b-e54c0665264c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:51.982033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165bc0c-1f0d-41a2-862b-e54c0665264c None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:51.986453 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae93627-63c6-4540-a7a8-ea5167604d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:51.987076 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae93627-63c6-4540-a7a8-ea5167604d8b service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:51.987348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae93627-63c6-4540-a7a8-ea5167604d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:51.987612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae93627-63c6-4540-a7a8-ea5167604d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:51.999030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae93627-63c6-4540-a7a8-ea5167604d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:52.000525 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae93627-63c6-4540-a7a8-ea5167604d8b service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:52.011528 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1113/2215] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:56.113754 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a44eb-1dd7-4465-9944-8b2bcb71f3a1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:56.116193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7d2a44eb-1dd7-4465-9944-8b2bcb71f3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:56.187148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a44eb-1dd7-4465-9944-8b2bcb71f3a1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:56.191971 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29c732-3178-4a93-b2d1-91389f2e9794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:56.192627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29c732-3178-4a93-b2d1-91389f2e9794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:56.192900 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29c732-3178-4a93-b2d1-91389f2e9794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:56.193171 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29c732-3178-4a93-b2d1-91389f2e9794 admin admin] PATH_INFO: `/v3/users/edfe17b405664e96bbeb149dc9c38378` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:56.194406 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29c732-3178-4a93-b2d1-91389f2e9794 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edfe17b405664e96bbeb149dc9c38378)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:56.196333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29c732-3178-4a93-b2d1-91389f2e9794 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:56.198970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:56.252042 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6d29c732-3178-4a93-b2d1-91389f2e9794 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edfe17b405664e96bbeb149dc9c38378'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:56.252717 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6d29c732-3178-4a93-b2d1-91389f2e9794 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edfe17b405664e96bbeb149dc9c38378'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:56.265696 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6d29c732-3178-4a93-b2d1-91389f2e9794 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edfe17b405664e96bbeb149dc9c38378'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:56.275166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6d29c732-3178-4a93-b2d1-91389f2e9794 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edfe17b405664e96bbeb149dc9c38378'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:56.282946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6d29c732-3178-4a93-b2d1-91389f2e9794 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edfe17b405664e96bbeb149dc9c38378'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:56.289271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1103/2216] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:26:56 2022] DELETE /identity/v3/users/edfe17b405664e96bbeb149dc9c38378 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:56.293913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455780a-5b02-4ef4-9f8b-ee371bd79951 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:56.294412 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455780a-5b02-4ef4-9f8b-ee371bd79951 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:56.294615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455780a-5b02-4ef4-9f8b-ee371bd79951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:56.313976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-a455780a-5b02-4ef4-9f8b-ee371bd79951 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:56.315218 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-a455780a-5b02-4ef4-9f8b-ee371bd79951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:56.383902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1114/2217] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:56.396921 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29c732-3178-4a93-b2d1-91389f2e9794 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:56.398573 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6d29c732-3178-4a93-b2d1-91389f2e9794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:56.458227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29c732-3178-4a93-b2d1-91389f2e9794 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:56.462136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df16e1-fadb-4436-8937-bc1f8196c6e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:56.462771 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df16e1-fadb-4436-8937-bc1f8196c6e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:56.463057 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df16e1-fadb-4436-8937-bc1f8196c6e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:56.463323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df16e1-fadb-4436-8937-bc1f8196c6e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:56.472654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df16e1-fadb-4436-8937-bc1f8196c6e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:56.474154 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df16e1-fadb-4436-8937-bc1f8196c6e8 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:56.484471 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1104/2218] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:26:56.781882 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455780a-5b02-4ef4-9f8b-ee371bd79951 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:56.783352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-a455780a-5b02-4ef4-9f8b-ee371bd79951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:56.795626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455780a-5b02-4ef4-9f8b-ee371bd79951 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:56.801413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb882d-4ba6-4f6d-b087-29ee598e615c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:56.802124 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb882d-4ba6-4f6d-b087-29ee598e615c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:56.802473 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb882d-4ba6-4f6d-b087-29ee598e615c admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:56.802744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb882d-4ba6-4f6d-b087-29ee598e615c admin admin] PATH_INFO: `/v3/projects/d00affda994040aeb2654ade7575502b` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:56.804067 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb882d-4ba6-4f6d-b087-29ee598e615c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d00affda994040aeb2654ade7575502b)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:56.806097 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb882d-4ba6-4f6d-b087-29ee598e615c admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:56.848410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:56.860740 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6ddb882d-4ba6-4f6d-b087-29ee598e615c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd00affda994040aeb2654ade7575502b', 'request_id': 'req-6ddb882d-4ba6-4f6d-b087-29ee598e615c'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:56.875720 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6ddb882d-4ba6-4f6d-b087-29ee598e615c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd00affda994040aeb2654ade7575502b', 'request_id': 'req-6ddb882d-4ba6-4f6d-b087-29ee598e615c'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:56.882808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6ddb882d-4ba6-4f6d-b087-29ee598e615c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd00affda994040aeb2654ade7575502b', 'request_id': 'req-6ddb882d-4ba6-4f6d-b087-29ee598e615c'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:56.895025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6ddb882d-4ba6-4f6d-b087-29ee598e615c admin admin] The token cache is being invalidate because project d00affda994040aeb2654ade7575502b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:56.895309 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6ddb882d-4ba6-4f6d-b087-29ee598e615c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:56.895952 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6ddb882d-4ba6-4f6d-b087-29ee598e615c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:56.898140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1115/2219] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:26:56 2022] DELETE /identity/v3/projects/d00affda994040aeb2654ade7575502b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:56.902527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c80eca-4c7a-4ff1-a306-ca2c31e3c2cd None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:56.902882 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c80eca-4c7a-4ff1-a306-ca2c31e3c2cd None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:56.903210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c80eca-4c7a-4ff1-a306-ca2c31e3c2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:56.924380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e0c80eca-4c7a-4ff1-a306-ca2c31e3c2cd None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:56.926277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e0c80eca-4c7a-4ff1-a306-ca2c31e3c2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:56.999081 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1105/2220] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:57.008317 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe6c76-8581-4ec8-a55b-d97759eedf24 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:57.008718 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe6c76-8581-4ec8-a55b-d97759eedf24 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:57.009210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe6c76-8581-4ec8-a55b-d97759eedf24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:57.031397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-efbe6c76-8581-4ec8-a55b-d97759eedf24 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:57.032754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-efbe6c76-8581-4ec8-a55b-d97759eedf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:57.041620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1116/2221] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:57.046146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c80eca-4c7a-4ff1-a306-ca2c31e3c2cd None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:57.056251 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c80eca-4c7a-4ff1-a306-ca2c31e3c2cd None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:57.061439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392779e-b2c8-4458-8ddd-8af52bcaa1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:57.062135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392779e-b2c8-4458-8ddd-8af52bcaa1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:57.062396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392779e-b2c8-4458-8ddd-8af52bcaa1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:57.062670 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392779e-b2c8-4458-8ddd-8af52bcaa1d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:57.063534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392779e-b2c8-4458-8ddd-8af52bcaa1d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:26:57.063817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392779e-b2c8-4458-8ddd-8af52bcaa1d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:57.065461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392779e-b2c8-4458-8ddd-8af52bcaa1d5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:57.077495 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1106/2222] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:26:57.082059 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe6c76-8581-4ec8-a55b-d97759eedf24 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:57.092172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe6c76-8581-4ec8-a55b-d97759eedf24 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:57.098092 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5c84d-2b80-4220-af4b-52574f968282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:57.098764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5c84d-2b80-4220-af4b-52574f968282 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:57.099101 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5c84d-2b80-4220-af4b-52574f968282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:57.099442 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5c84d-2b80-4220-af4b-52574f968282 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:57.100176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e5c84d-2b80-4220-af4b-52574f968282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:57.102207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e5c84d-2b80-4220-af4b-52574f968282 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:57.102733 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94ad74-ee3e-445a-92c3-9bb3a065d465 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:57.103028 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94ad74-ee3e-445a-92c3-9bb3a065d465 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:57.103438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94ad74-ee3e-445a-92c3-9bb3a065d465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:57.127999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5d94ad74-ee3e-445a-92c3-9bb3a065d465 None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:57.129304 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5d94ad74-ee3e-445a-92c3-9bb3a065d465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:57.135705 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:57.149086 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1117/2223] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:26:57 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:26:57.150804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5c84d-2b80-4220-af4b-52574f968282 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:57.192207 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1107/2224] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 15:26:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:26:57.215231 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94ad74-ee3e-445a-92c3-9bb3a065d465 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:57.215782 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5c84d-2b80-4220-af4b-52574f968282 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:57.216773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5d94ad74-ee3e-445a-92c3-9bb3a065d465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:57.221298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bdcc0-1dc3-4389-b902-fa7a0bc3498e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:57.221963 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bdcc0-1dc3-4389-b902-fa7a0bc3498e admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:57.222241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bdcc0-1dc3-4389-b902-fa7a0bc3498e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:57.222508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bdcc0-1dc3-4389-b902-fa7a0bc3498e admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:57.223179 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7bdcc0-1dc3-4389-b902-fa7a0bc3498e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:57.225109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7bdcc0-1dc3-4389-b902-fa7a0bc3498e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:57.247646 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1118/2225] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:26:57 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:57.252251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bdcc0-1dc3-4389-b902-fa7a0bc3498e admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:57.265167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bdcc0-1dc3-4389-b902-fa7a0bc3498e admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:57.270289 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb08dd8-e13b-4c99-8c84-b7543e4bbe02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:57.270920 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb08dd8-e13b-4c99-8c84-b7543e4bbe02 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:57.271199 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb08dd8-e13b-4c99-8c84-b7543e4bbe02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:57.271472 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb08dd8-e13b-4c99-8c84-b7543e4bbe02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:57.272182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb08dd8-e13b-4c99-8c84-b7543e4bbe02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:57.273803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb08dd8-e13b-4c99-8c84-b7543e4bbe02 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:57.288005 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1119/2226] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:26:57.289194 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb08dd8-e13b-4c99-8c84-b7543e4bbe02 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:57.296782 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94ad74-ee3e-445a-92c3-9bb3a065d465 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:57.299581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb08dd8-e13b-4c99-8c84-b7543e4bbe02 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:57.304258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7c31e-97f1-4e99-a762-4b9587b0da4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:57.304877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7c31e-97f1-4e99-a762-4b9587b0da4e admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:57.305154 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7c31e-97f1-4e99-a762-4b9587b0da4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:57.305398 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7c31e-97f1-4e99-a762-4b9587b0da4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:57.306143 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7c31e-97f1-4e99-a762-4b9587b0da4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:57.307756 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7c31e-97f1-4e99-a762-4b9587b0da4e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:57.318751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1120/2227] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:26:57.323136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7c31e-97f1-4e99-a762-4b9587b0da4e admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:57.334381 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7c31e-97f1-4e99-a762-4b9587b0da4e admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:57.340118 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ea466-3810-4427-92be-a52a7af6ac7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:57.340890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ea466-3810-4427-92be-a52a7af6ac7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:57.341171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ea466-3810-4427-92be-a52a7af6ac7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:57.341444 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ea466-3810-4427-92be-a52a7af6ac7c admin admin] PATH_INFO: `/v3/projects/f419bba61dd846daae1483364412b5f4/users/b1d3bd584c074be5a934d3f31fd411b7/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:57.357208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ea466-3810-4427-92be-a52a7af6ac7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f419bba61dd846daae1483364412b5f4, user_id=b1d3bd584c074be5a934d3f31fd411b7, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:57.360225 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ea466-3810-4427-92be-a52a7af6ac7c admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:57.364341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a403987-1b7e-47ed-a749-164156306d24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:57.364977 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a403987-1b7e-47ed-a749-164156306d24 service swift] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:57.365253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a403987-1b7e-47ed-a749-164156306d24 service swift] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:57.365526 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a403987-1b7e-47ed-a749-164156306d24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:57.378896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1121/2228] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:26:57 2022] PUT /identity/v3/projects/f419bba61dd846daae1483364412b5f4/users/b1d3bd584c074be5a934d3f31fd411b7/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:26:57.390809 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0d6c7-26f6-4804-ba6a-9c79b30a3c0d None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:57.391736 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0d6c7-26f6-4804-ba6a-9c79b30a3c0d None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:57.392079 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0d6c7-26f6-4804-ba6a-9c79b30a3c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:57.430727 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-28d0d6c7-26f6-4804-ba6a-9c79b30a3c0d None None] MFA Rules not processed for user `b1d3bd584c074be5a934d3f31fd411b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:57.433350 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-28d0d6c7-26f6-4804-ba6a-9c79b30a3c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:57.447229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a403987-1b7e-47ed-a749-164156306d24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:57.449528 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a403987-1b7e-47ed-a749-164156306d24 service swift] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:57.456207 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:57.468190 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1108/2229] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:26:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:26:57.469465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:DELETE=5,keystone:SELECT=89,keystone:INSERT=6 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:57.524220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1122/2230] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:26:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:57.538159 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a403987-1b7e-47ed-a749-164156306d24 service swift] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:57.539644 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3a403987-1b7e-47ed-a749-164156306d24 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:57.633131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a403987-1b7e-47ed-a749-164156306d24 service swift] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:57.763514 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6efa8-c172-4709-a1f5-7321b3973f00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:57.764078 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6efa8-c172-4709-a1f5-7321b3973f00 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:57.764360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6efa8-c172-4709-a1f5-7321b3973f00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:57.764642 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6efa8-c172-4709-a1f5-7321b3973f00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:57.785574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6efa8-c172-4709-a1f5-7321b3973f00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:57.787108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6efa8-c172-4709-a1f5-7321b3973f00 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:57.815025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1109/2231] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:26:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:00.133465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0d6c7-26f6-4804-ba6a-9c79b30a3c0d None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:00.134983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-28d0d6c7-26f6-4804-ba6a-9c79b30a3c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:00.209352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0d6c7-26f6-4804-ba6a-9c79b30a3c0d None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:00.214700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af9ac7-884a-4edd-8606-9e0f288464f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:00.215368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af9ac7-884a-4edd-8606-9e0f288464f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:00.215650 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af9ac7-884a-4edd-8606-9e0f288464f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:00.215924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af9ac7-884a-4edd-8606-9e0f288464f8 admin admin] PATH_INFO: `/v3/users/3ac66f9b870c44e287b21d6cc5e3f139` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:00.217344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af9ac7-884a-4edd-8606-9e0f288464f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ac66f9b870c44e287b21d6cc5e3f139)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:00.219398 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af9ac7-884a-4edd-8606-9e0f288464f8 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:00.271045 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-30af9ac7-884a-4edd-8606-9e0f288464f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ac66f9b870c44e287b21d6cc5e3f139'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:00.279167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-30af9ac7-884a-4edd-8606-9e0f288464f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ac66f9b870c44e287b21d6cc5e3f139'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:00.280144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-30af9ac7-884a-4edd-8606-9e0f288464f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ac66f9b870c44e287b21d6cc5e3f139'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:00.287475 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-30af9ac7-884a-4edd-8606-9e0f288464f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ac66f9b870c44e287b21d6cc5e3f139'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:00.300291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-30af9ac7-884a-4edd-8606-9e0f288464f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ac66f9b870c44e287b21d6cc5e3f139'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:00.306527 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1123/2232] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:27:00 2022] DELETE /identity/v3/users/3ac66f9b870c44e287b21d6cc5e3f139 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:00.311271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6d956-63ea-4c00-877d-17e87f8a2768 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:00.311821 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6d956-63ea-4c00-877d-17e87f8a2768 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:00.311821 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6d956-63ea-4c00-877d-17e87f8a2768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:00.332655 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7ad6d956-63ea-4c00-877d-17e87f8a2768 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:00.334049 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7ad6d956-63ea-4c00-877d-17e87f8a2768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:00.427303 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1110/2233] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:00.437329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af9ac7-884a-4edd-8606-9e0f288464f8 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:00.439055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-30af9ac7-884a-4edd-8606-9e0f288464f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:00.494642 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:00.501209 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af9ac7-884a-4edd-8606-9e0f288464f8 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:00.505910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34267858-9fab-45ff-8e31-d8d159dab701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:00.506568 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34267858-9fab-45ff-8e31-d8d159dab701 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:00.507276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34267858-9fab-45ff-8e31-d8d159dab701 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:00.507778 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34267858-9fab-45ff-8e31-d8d159dab701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:00.518042 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34267858-9fab-45ff-8e31-d8d159dab701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:00.519496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34267858-9fab-45ff-8e31-d8d159dab701 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:00.529108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1124/2234] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:00.834485 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6d956-63ea-4c00-877d-17e87f8a2768 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:00.836525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7ad6d956-63ea-4c00-877d-17e87f8a2768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:00.850445 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad6d956-63ea-4c00-877d-17e87f8a2768 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:00.856131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a708e5-e75f-4f68-84fa-6feef6f97ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:00.856780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a708e5-e75f-4f68-84fa-6feef6f97ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:00.857331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a708e5-e75f-4f68-84fa-6feef6f97ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:00.857669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a708e5-e75f-4f68-84fa-6feef6f97ac7 admin admin] PATH_INFO: `/v3/projects/f4e32ad093cf41e082bcedd5072092d3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:00.860177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a708e5-e75f-4f68-84fa-6feef6f97ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e32ad093cf41e082bcedd5072092d3)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:00.862241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a708e5-e75f-4f68-84fa-6feef6f97ac7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:00.915623 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-23a708e5-e75f-4f68-84fa-6feef6f97ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e32ad093cf41e082bcedd5072092d3', 'request_id': 'req-23a708e5-e75f-4f68-84fa-6feef6f97ac7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:00.919435 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:00.921612 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-23a708e5-e75f-4f68-84fa-6feef6f97ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e32ad093cf41e082bcedd5072092d3', 'request_id': 'req-23a708e5-e75f-4f68-84fa-6feef6f97ac7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:00.930025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-23a708e5-e75f-4f68-84fa-6feef6f97ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e32ad093cf41e082bcedd5072092d3', 'request_id': 'req-23a708e5-e75f-4f68-84fa-6feef6f97ac7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:00.941844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-23a708e5-e75f-4f68-84fa-6feef6f97ac7 admin admin] The token cache is being invalidate because project f4e32ad093cf41e082bcedd5072092d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:27:00.942082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-23a708e5-e75f-4f68-84fa-6feef6f97ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:00.942720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-23a708e5-e75f-4f68-84fa-6feef6f97ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:00.944250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1111/2235] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:27:00 2022] DELETE /identity/v3/projects/f4e32ad093cf41e082bcedd5072092d3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:00.949807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e1c35-b3fb-48fd-bd43-27a92a409738 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:00.950134 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e1c35-b3fb-48fd-bd43-27a92a409738 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:00.950455 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e1c35-b3fb-48fd-bd43-27a92a409738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:00.976434 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-fe8e1c35-b3fb-48fd-bd43-27a92a409738 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:00.977643 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-fe8e1c35-b3fb-48fd-bd43-27a92a409738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:01.060422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1125/2236] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:01.068548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676af27d-9e67-4aaf-92ff-cb290acc75fa None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:01.069179 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676af27d-9e67-4aaf-92ff-cb290acc75fa None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:01.069620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676af27d-9e67-4aaf-92ff-cb290acc75fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:01.091517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-676af27d-9e67-4aaf-92ff-cb290acc75fa None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:01.093166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-676af27d-9e67-4aaf-92ff-cb290acc75fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:01.102747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1112/2237] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:01.107950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e1c35-b3fb-48fd-bd43-27a92a409738 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:01.120434 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e1c35-b3fb-48fd-bd43-27a92a409738 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:01.125901 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12668a1-2060-4638-b231-755eacd34c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:01.126538 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12668a1-2060-4638-b231-755eacd34c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:01.126830 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12668a1-2060-4638-b231-755eacd34c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:01.127161 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12668a1-2060-4638-b231-755eacd34c7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:01.127786 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12668a1-2060-4638-b231-755eacd34c7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:27:01.128087 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12668a1-2060-4638-b231-755eacd34c7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:01.129732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12668a1-2060-4638-b231-755eacd34c7f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:01.142923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1126/2238] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:27:01.147326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676af27d-9e67-4aaf-92ff-cb290acc75fa None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:01.158372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676af27d-9e67-4aaf-92ff-cb290acc75fa None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:01.163061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa3bbc-8864-46af-aa39-d2d893c860c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:01.163901 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa3bbc-8864-46af-aa39-d2d893c860c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:01.164181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa3bbc-8864-46af-aa39-d2d893c860c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:01.165048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa3bbc-8864-46af-aa39-d2d893c860c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:01.165793 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa3bbc-8864-46af-aa39-d2d893c860c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:01.168350 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa3bbc-8864-46af-aa39-d2d893c860c2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:01.223625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1113/2239] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:27:01 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:27:01.227761 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12668a1-2060-4638-b231-755eacd34c7f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:01.251520 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12668a1-2060-4638-b231-755eacd34c7f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:01.352232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151d2e5-4ae6-485f-b2e3-535f83711afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:01.352875 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151d2e5-4ae6-485f-b2e3-535f83711afc admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:01.353154 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151d2e5-4ae6-485f-b2e3-535f83711afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:01.353537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151d2e5-4ae6-485f-b2e3-535f83711afc admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:01.354378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151d2e5-4ae6-485f-b2e3-535f83711afc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:01.356374 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151d2e5-4ae6-485f-b2e3-535f83711afc admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:01.387866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1127/2240] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:27:01 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:27:01.389379 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa3bbc-8864-46af-aa39-d2d893c860c2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:01.400587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa3bbc-8864-46af-aa39-d2d893c860c2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:01.409050 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5a215-8654-45f7-be9b-bbdb3052167e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:01.409823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5a215-8654-45f7-be9b-bbdb3052167e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:01.410164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5a215-8654-45f7-be9b-bbdb3052167e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:01.410509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5a215-8654-45f7-be9b-bbdb3052167e admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:01.411286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5a215-8654-45f7-be9b-bbdb3052167e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:01.413108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5a215-8654-45f7-be9b-bbdb3052167e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:01.429558 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1114/2241] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:01.434131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151d2e5-4ae6-485f-b2e3-535f83711afc admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:01.455900 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151d2e5-4ae6-485f-b2e3-535f83711afc admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:01.462719 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3b673-a5c5-4b0e-96ba-adcc4bd4ea78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:01.464223 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3b673-a5c5-4b0e-96ba-adcc4bd4ea78 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:01.464779 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3b673-a5c5-4b0e-96ba-adcc4bd4ea78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:01.465136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3b673-a5c5-4b0e-96ba-adcc4bd4ea78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:01.466725 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3b673-a5c5-4b0e-96ba-adcc4bd4ea78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:01.468371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3b673-a5c5-4b0e-96ba-adcc4bd4ea78 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:01.490806 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1128/2242] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:01.492125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5a215-8654-45f7-be9b-bbdb3052167e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:01.507099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5a215-8654-45f7-be9b-bbdb3052167e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:01.511843 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8198f31-75f1-4b25-b662-57f95a836a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:01.519983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8198f31-75f1-4b25-b662-57f95a836a5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:01.519983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8198f31-75f1-4b25-b662-57f95a836a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:01.519983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8198f31-75f1-4b25-b662-57f95a836a5a admin admin] PATH_INFO: `/v3/projects/4ad998384c894fe9aed6d57f7aa1110b/users/30b9685e174a4523a6c36d4d0bebd5fc/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:01.538846 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8198f31-75f1-4b25-b662-57f95a836a5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ad998384c894fe9aed6d57f7aa1110b, user_id=30b9685e174a4523a6c36d4d0bebd5fc, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:01.541627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8198f31-75f1-4b25-b662-57f95a836a5a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:01.556771 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1115/2243] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:27:01 2022] PUT /identity/v3/projects/4ad998384c894fe9aed6d57f7aa1110b/users/30b9685e174a4523a6c36d4d0bebd5fc/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:01.562173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a563f64-6f90-40a2-891b-511fddd98bee None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:01.562461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a563f64-6f90-40a2-891b-511fddd98bee None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:01.562872 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a563f64-6f90-40a2-891b-511fddd98bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:01.582584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-8a563f64-6f90-40a2-891b-511fddd98bee None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:01.583808 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8a563f64-6f90-40a2-891b-511fddd98bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:01.633933 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:01.650944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1129/2244] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:01.663395 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8198f31-75f1-4b25-b662-57f95a836a5a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:01.665350 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d8198f31-75f1-4b25-b662-57f95a836a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:01.737059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8198f31-75f1-4b25-b662-57f95a836a5a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:01.741212 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605b555-3f29-4f9d-af94-9dcd486589dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:01.741956 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605b555-3f29-4f9d-af94-9dcd486589dc service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:01.742302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605b555-3f29-4f9d-af94-9dcd486589dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:01.742638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605b555-3f29-4f9d-af94-9dcd486589dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:01.754118 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605b555-3f29-4f9d-af94-9dcd486589dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:01.756084 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605b555-3f29-4f9d-af94-9dcd486589dc service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:01.768027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1116/2245] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:07.321474 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab866a3-6ba6-426f-b727-ff8a20979838 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:07.322083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab866a3-6ba6-426f-b727-ff8a20979838 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:07.322083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab866a3-6ba6-426f-b727-ff8a20979838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:07.392055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-eab866a3-6ba6-426f-b727-ff8a20979838 None None] MFA Rules not processed for user `30b9685e174a4523a6c36d4d0bebd5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:07.393320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-eab866a3-6ba6-426f-b727-ff8a20979838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:07.482378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1130/2246] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:27:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:07.486802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605b555-3f29-4f9d-af94-9dcd486589dc service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:07.497340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605b555-3f29-4f9d-af94-9dcd486589dc service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:07.502545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a8ebe-f033-4627-a59b-1abb67bb7022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:07.503185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a8ebe-f033-4627-a59b-1abb67bb7022 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:07.503456 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a8ebe-f033-4627-a59b-1abb67bb7022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:07.503723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a8ebe-f033-4627-a59b-1abb67bb7022 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:07.504398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a8ebe-f033-4627-a59b-1abb67bb7022 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:07.506426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a8ebe-f033-4627-a59b-1abb67bb7022 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:07.583945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1117/2247] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:27:07 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:27:07.590661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab866a3-6ba6-426f-b727-ff8a20979838 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:07.621581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab866a3-6ba6-426f-b727-ff8a20979838 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:07.743983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe0891-c297-4526-8407-14ce33e34e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:07.744627 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe0891-c297-4526-8407-14ce33e34e6a admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:07.744905 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe0891-c297-4526-8407-14ce33e34e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:07.745178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe0891-c297-4526-8407-14ce33e34e6a admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:07.746160 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe0891-c297-4526-8407-14ce33e34e6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:07.748142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe0891-c297-4526-8407-14ce33e34e6a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:07.780807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1131/2248] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:27:07 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:27:07.784830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a8ebe-f033-4627-a59b-1abb67bb7022 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:07.804601 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a8ebe-f033-4627-a59b-1abb67bb7022 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:07.819827 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234212c-0912-44fa-83e4-9d2b02613808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:07.820486 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234212c-0912-44fa-83e4-9d2b02613808 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:07.820769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234212c-0912-44fa-83e4-9d2b02613808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:07.821037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234212c-0912-44fa-83e4-9d2b02613808 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:07.821755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234212c-0912-44fa-83e4-9d2b02613808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:07.823444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234212c-0912-44fa-83e4-9d2b02613808 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:07.846748 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1118/2249] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:07.850692 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe0891-c297-4526-8407-14ce33e34e6a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:07.862274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe0891-c297-4526-8407-14ce33e34e6a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:07.871698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b899f-9e18-41c2-9e09-0e76160a87a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:07.873191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b899f-9e18-41c2-9e09-0e76160a87a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:07.873585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b899f-9e18-41c2-9e09-0e76160a87a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:07.873989 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b899f-9e18-41c2-9e09-0e76160a87a2 admin admin] PATH_INFO: `/v3/projects/a8d4f982f0664ab18c1a6352d0c07c66/users/d79de80f35634aa78e04a5765ae8f99f/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:07.894396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b899f-9e18-41c2-9e09-0e76160a87a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d4f982f0664ab18c1a6352d0c07c66, user_id=d79de80f35634aa78e04a5765ae8f99f, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:07.897065 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b899f-9e18-41c2-9e09-0e76160a87a2 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:07.912119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1132/2250] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:27:07 2022] PUT /identity/v3/projects/a8d4f982f0664ab18c1a6352d0c07c66/users/d79de80f35634aa78e04a5765ae8f99f/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:27:07.916089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234212c-0912-44fa-83e4-9d2b02613808 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:07.926868 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234212c-0912-44fa-83e4-9d2b02613808 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:07.976507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:08.006462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff5c04-2afd-400c-8064-2dc3ffb3e73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:08.007061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5c04-2afd-400c-8064-2dc3ffb3e73b admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:08.007344 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5c04-2afd-400c-8064-2dc3ffb3e73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:08.007616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5c04-2afd-400c-8064-2dc3ffb3e73b admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:08.008412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff5c04-2afd-400c-8064-2dc3ffb3e73b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:08.009983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff5c04-2afd-400c-8064-2dc3ffb3e73b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:08.026029 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1119/2251] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:08.027550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b899f-9e18-41c2-9e09-0e76160a87a2 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:08.042991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b899f-9e18-41c2-9e09-0e76160a87a2 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:08.055711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb5a33-0a53-4edc-a0b6-8eed1126acd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:08.056376 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb5a33-0a53-4edc-a0b6-8eed1126acd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:08.058650 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb5a33-0a53-4edc-a0b6-8eed1126acd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:08.058941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb5a33-0a53-4edc-a0b6-8eed1126acd6 admin admin] PATH_INFO: `/v3/projects/a8d4f982f0664ab18c1a6352d0c07c66/users/d79de80f35634aa78e04a5765ae8f99f/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:08.067565 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb5a33-0a53-4edc-a0b6-8eed1126acd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d4f982f0664ab18c1a6352d0c07c66, user_id=d79de80f35634aa78e04a5765ae8f99f, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:08.070602 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb5a33-0a53-4edc-a0b6-8eed1126acd6 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:08.101578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1133/2252] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:27:08 2022] PUT /identity/v3/projects/a8d4f982f0664ab18c1a6352d0c07c66/users/d79de80f35634aa78e04a5765ae8f99f/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:09.673231 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff5c04-2afd-400c-8064-2dc3ffb3e73b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:09.675277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-96ff5c04-2afd-400c-8064-2dc3ffb3e73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:09.698602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff5c04-2afd-400c-8064-2dc3ffb3e73b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:09.802843 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409eed7-9096-439b-baaf-bdd125f3ea91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:09.803810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409eed7-9096-439b-baaf-bdd125f3ea91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:09.804444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409eed7-9096-439b-baaf-bdd125f3ea91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:09.805266 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409eed7-9096-439b-baaf-bdd125f3ea91 admin admin] PATH_INFO: `/v3/users/b1d3bd584c074be5a934d3f31fd411b7` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:09.807003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409eed7-9096-439b-baaf-bdd125f3ea91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1d3bd584c074be5a934d3f31fd411b7)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:09.809378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409eed7-9096-439b-baaf-bdd125f3ea91 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:09.882656 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4409eed7-9096-439b-baaf-bdd125f3ea91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1d3bd584c074be5a934d3f31fd411b7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:09.883736 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4409eed7-9096-439b-baaf-bdd125f3ea91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1d3bd584c074be5a934d3f31fd411b7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:09.903203 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4409eed7-9096-439b-baaf-bdd125f3ea91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1d3bd584c074be5a934d3f31fd411b7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:09.916641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4409eed7-9096-439b-baaf-bdd125f3ea91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1d3bd584c074be5a934d3f31fd411b7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:09.928733 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4409eed7-9096-439b-baaf-bdd125f3ea91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1d3bd584c074be5a934d3f31fd411b7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:09.943485 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1120/2253] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:27:09 2022] DELETE /identity/v3/users/b1d3bd584c074be5a934d3f31fd411b7 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:09.947762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccf7aa-9a72-4e7a-b915-c7e21a96d517 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:09.948671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccf7aa-9a72-4e7a-b915-c7e21a96d517 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:09.949101 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ccf7aa-9a72-4e7a-b915-c7e21a96d517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:09.995003 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-05ccf7aa-9a72-4e7a-b915-c7e21a96d517 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:09.998675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-05ccf7aa-9a72-4e7a-b915-c7e21a96d517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:10.068522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:10.112215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1134/2254] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:10.132536 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409eed7-9096-439b-baaf-bdd125f3ea91 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:10.134125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4409eed7-9096-439b-baaf-bdd125f3ea91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:10.368944 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409eed7-9096-439b-baaf-bdd125f3ea91 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:10.381125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5eed23-043d-481b-aecc-482760f65420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:10.381806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5eed23-043d-481b-aecc-482760f65420 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:10.382089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5eed23-043d-481b-aecc-482760f65420 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:10.382394 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5eed23-043d-481b-aecc-482760f65420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:10.398983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5eed23-043d-481b-aecc-482760f65420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:10.400500 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5eed23-043d-481b-aecc-482760f65420 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:10.427151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1121/2255] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:11.209178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccf7aa-9a72-4e7a-b915-c7e21a96d517 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:11.211273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-05ccf7aa-9a72-4e7a-b915-c7e21a96d517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:11.229663 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ccf7aa-9a72-4e7a-b915-c7e21a96d517 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:11.234694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46c73b-7435-49d5-b2b8-89ef14831086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:11.235610 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46c73b-7435-49d5-b2b8-89ef14831086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:11.236305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46c73b-7435-49d5-b2b8-89ef14831086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:11.236736 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46c73b-7435-49d5-b2b8-89ef14831086 admin admin] PATH_INFO: `/v3/projects/f419bba61dd846daae1483364412b5f4` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:11.238273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46c73b-7435-49d5-b2b8-89ef14831086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f419bba61dd846daae1483364412b5f4)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:11.241714 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46c73b-7435-49d5-b2b8-89ef14831086 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:11.302564 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ae46c73b-7435-49d5-b2b8-89ef14831086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f419bba61dd846daae1483364412b5f4', 'request_id': 'req-ae46c73b-7435-49d5-b2b8-89ef14831086'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:11.314343 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ae46c73b-7435-49d5-b2b8-89ef14831086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f419bba61dd846daae1483364412b5f4', 'request_id': 'req-ae46c73b-7435-49d5-b2b8-89ef14831086'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:11.321726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ae46c73b-7435-49d5-b2b8-89ef14831086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f419bba61dd846daae1483364412b5f4', 'request_id': 'req-ae46c73b-7435-49d5-b2b8-89ef14831086'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:11.336146 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ae46c73b-7435-49d5-b2b8-89ef14831086 admin admin] The token cache is being invalidate because project f419bba61dd846daae1483364412b5f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:27:11.336146 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ae46c73b-7435-49d5-b2b8-89ef14831086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:11.336753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ae46c73b-7435-49d5-b2b8-89ef14831086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:11.338667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1135/2256] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:27:11 2022] DELETE /identity/v3/projects/f419bba61dd846daae1483364412b5f4 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:11.344062 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f8116-2d3e-4a63-9597-ff205e204392 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:11.344625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f8116-2d3e-4a63-9597-ff205e204392 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:11.344625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f8116-2d3e-4a63-9597-ff205e204392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:11.363681 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:11.375517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ab1f8116-2d3e-4a63-9597-ff205e204392 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:11.377087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ab1f8116-2d3e-4a63-9597-ff205e204392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:11.467444 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1122/2257] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:11.470406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562194b-e796-4a36-a37e-dedd80bc2de6 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:11.470931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562194b-e796-4a36-a37e-dedd80bc2de6 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:11.471383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562194b-e796-4a36-a37e-dedd80bc2de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:11.519436 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-5562194b-e796-4a36-a37e-dedd80bc2de6 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:11.520860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5562194b-e796-4a36-a37e-dedd80bc2de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:11.545106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1136/2258] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:11.547984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f8116-2d3e-4a63-9597-ff205e204392 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:11.589836 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f8116-2d3e-4a63-9597-ff205e204392 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:11.603985 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9326406-5d26-4eeb-9694-a61c91cedfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:11.604812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9326406-5d26-4eeb-9694-a61c91cedfbe admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:11.605095 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9326406-5d26-4eeb-9694-a61c91cedfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:11.605364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9326406-5d26-4eeb-9694-a61c91cedfbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:11.606336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9326406-5d26-4eeb-9694-a61c91cedfbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:27:11.606623 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9326406-5d26-4eeb-9694-a61c91cedfbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:11.608494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9326406-5d26-4eeb-9694-a61c91cedfbe admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:11.639276 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1123/2259] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:27:11.643958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562194b-e796-4a36-a37e-dedd80bc2de6 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:11.656964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562194b-e796-4a36-a37e-dedd80bc2de6 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:11.669839 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f21e5-6080-4fc5-ae0f-aae57588e8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:11.669839 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f21e5-6080-4fc5-ae0f-aae57588e8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:11.669839 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f21e5-6080-4fc5-ae0f-aae57588e8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:11.669839 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f21e5-6080-4fc5-ae0f-aae57588e8b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:11.670685 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f21e5-6080-4fc5-ae0f-aae57588e8b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:11.670785 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697f21e5-6080-4fc5-ae0f-aae57588e8b0 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:11.733960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1137/2260] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:27:11 2022] POST /identity/v3/projects => generated 381 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:27:11.735125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9326406-5d26-4eeb-9694-a61c91cedfbe admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:11.751625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9326406-5d26-4eeb-9694-a61c91cedfbe admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:11.897588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f2dbb-e4ce-49a2-b059-769a78b6fb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:11.898216 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f2dbb-e4ce-49a2-b059-769a78b6fb81 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:11.898503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f2dbb-e4ce-49a2-b059-769a78b6fb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:11.898778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f2dbb-e4ce-49a2-b059-769a78b6fb81 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:11.899586 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f2dbb-e4ce-49a2-b059-769a78b6fb81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:11.901452 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f2dbb-e4ce-49a2-b059-769a78b6fb81 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:11.940303 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1124/2261] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:27:11 2022] POST /identity/v3/users => generated 291 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:27:11.944378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f21e5-6080-4fc5-ae0f-aae57588e8b0 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:11.958492 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f21e5-6080-4fc5-ae0f-aae57588e8b0 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:11.968977 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2036116-b824-41c2-82ad-779197855d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:11.970362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2036116-b824-41c2-82ad-779197855d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:11.970649 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2036116-b824-41c2-82ad-779197855d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:11.970927 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2036116-b824-41c2-82ad-779197855d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:11.971662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2036116-b824-41c2-82ad-779197855d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:11.977587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2036116-b824-41c2-82ad-779197855d24 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:12.010208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1138/2262] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:12.014021 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f2dbb-e4ce-49a2-b059-769a78b6fb81 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:12.032517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f2dbb-e4ce-49a2-b059-769a78b6fb81 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:12.044630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d52c6-657d-49f7-a27d-5f7a943bc804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:12.045772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d52c6-657d-49f7-a27d-5f7a943bc804 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:12.046750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d52c6-657d-49f7-a27d-5f7a943bc804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:12.047190 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d52c6-657d-49f7-a27d-5f7a943bc804 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:12.048074 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d52c6-657d-49f7-a27d-5f7a943bc804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:12.050362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d52c6-657d-49f7-a27d-5f7a943bc804 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:12.086199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1125/2263] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:12.086899 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2036116-b824-41c2-82ad-779197855d24 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:12.105175 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2036116-b824-41c2-82ad-779197855d24 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:12.112582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce14772-a66c-44aa-bb2a-57bac9786a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:12.113653 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce14772-a66c-44aa-bb2a-57bac9786a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:12.114111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce14772-a66c-44aa-bb2a-57bac9786a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:12.114569 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce14772-a66c-44aa-bb2a-57bac9786a17 admin admin] PATH_INFO: `/v3/projects/dc512d8e2ab848fda424f9eeb6c816c9/users/b1afe2bda6784f9987b8aa5fa174dde2/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:12.136110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce14772-a66c-44aa-bb2a-57bac9786a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc512d8e2ab848fda424f9eeb6c816c9, user_id=b1afe2bda6784f9987b8aa5fa174dde2, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:12.140011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce14772-a66c-44aa-bb2a-57bac9786a17 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:12.157695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1139/2264] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:27:12 2022] PUT /identity/v3/projects/dc512d8e2ab848fda424f9eeb6c816c9/users/b1afe2bda6784f9987b8aa5fa174dde2/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:27:12.171661 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c30279-7d96-4ce6-9df5-f3c3f3bc48ea None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:12.171957 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c30279-7d96-4ce6-9df5-f3c3f3bc48ea None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:12.172209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c30279-7d96-4ce6-9df5-f3c3f3bc48ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:12.217489 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c3c30279-7d96-4ce6-9df5-f3c3f3bc48ea None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:12.218888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c3c30279-7d96-4ce6-9df5-f3c3f3bc48ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:12.331844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1126/2265] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:12.343158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce14772-a66c-44aa-bb2a-57bac9786a17 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:12.345150 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1ce14772-a66c-44aa-bb2a-57bac9786a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:12.481398 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce14772-a66c-44aa-bb2a-57bac9786a17 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:12.487053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114df1f-37a5-4988-a8db-46cdf39b9179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:12.487781 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114df1f-37a5-4988-a8db-46cdf39b9179 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:12.488057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114df1f-37a5-4988-a8db-46cdf39b9179 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:12.488331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114df1f-37a5-4988-a8db-46cdf39b9179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:12.502163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114df1f-37a5-4988-a8db-46cdf39b9179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:12.504029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114df1f-37a5-4988-a8db-46cdf39b9179 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:12.530987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:12.564171 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1140/2266] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:13.831341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5000dd-a2af-44c0-a23c-ab0a81f4cf61 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:13.831938 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5000dd-a2af-44c0-a23c-ab0a81f4cf61 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:13.831938 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5000dd-a2af-44c0-a23c-ab0a81f4cf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:13.859111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:13.862299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-0f5000dd-a2af-44c0-a23c-ab0a81f4cf61 None None] MFA Rules not processed for user `b1afe2bda6784f9987b8aa5fa174dde2`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:13.863743 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0f5000dd-a2af-44c0-a23c-ab0a81f4cf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:13.961201 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1127/2267] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:27:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:13.982636 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114df1f-37a5-4988-a8db-46cdf39b9179 service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:13.984863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d114df1f-37a5-4988-a8db-46cdf39b9179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:14.083752 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114df1f-37a5-4988-a8db-46cdf39b9179 service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:14.090000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eaaaa3-f2f9-4ea7-916d-5f4caf75ae77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:14.091228 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eaaaa3-f2f9-4ea7-916d-5f4caf75ae77 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:14.091684 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eaaaa3-f2f9-4ea7-916d-5f4caf75ae77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:14.092110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eaaaa3-f2f9-4ea7-916d-5f4caf75ae77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:14.102035 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eaaaa3-f2f9-4ea7-916d-5f4caf75ae77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:14.103765 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eaaaa3-f2f9-4ea7-916d-5f4caf75ae77 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:14.115890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1141/2268] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:14.304987 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108103a-af6a-44fa-bf93-8d8d79cbcd10 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:14.305326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108103a-af6a-44fa-bf93-8d8d79cbcd10 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:14.305563 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108103a-af6a-44fa-bf93-8d8d79cbcd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:14.344951 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-8108103a-af6a-44fa-bf93-8d8d79cbcd10 None None] MFA Rules not processed for user `d79de80f35634aa78e04a5765ae8f99f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:14.346456 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8108103a-af6a-44fa-bf93-8d8d79cbcd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:14.461685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1128/2269] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:27:14 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:14.476607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eaaaa3-f2f9-4ea7-916d-5f4caf75ae77 service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:14.478119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f4eaaaa3-f2f9-4ea7-916d-5f4caf75ae77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:14.569540 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eaaaa3-f2f9-4ea7-916d-5f4caf75ae77 service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:14.576542 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ca989-2866-4a69-a55a-d15b39de6b4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:14.577198 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ca989-2866-4a69-a55a-d15b39de6b4d service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:14.577458 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ca989-2866-4a69-a55a-d15b39de6b4d service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:14.577824 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ca989-2866-4a69-a55a-d15b39de6b4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:14.580605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-811ca989-2866-4a69-a55a-d15b39de6b4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:14.649331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ca989-2866-4a69-a55a-d15b39de6b4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:14.651155 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811ca989-2866-4a69-a55a-d15b39de6b4d service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:14.662328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1142/2270] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:27:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:15.668897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e334a39-cc56-4aad-b4e2-62a4edfafe39 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:15.669429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e334a39-cc56-4aad-b4e2-62a4edfafe39 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:15.669751 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e334a39-cc56-4aad-b4e2-62a4edfafe39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:15.692317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2e334a39-cc56-4aad-b4e2-62a4edfafe39 None None] MFA Rules not processed for user `b1afe2bda6784f9987b8aa5fa174dde2`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:15.693755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2e334a39-cc56-4aad-b4e2-62a4edfafe39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:15.702236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1129/2271] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:27:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:17.326880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcfe27-21db-4df4-8763-b13c6413f6f4 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:17.327422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcfe27-21db-4df4-8763-b13c6413f6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:17.327700 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcfe27-21db-4df4-8763-b13c6413f6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:17.350934 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-07dcfe27-21db-4df4-8763-b13c6413f6f4 None None] MFA Rules not processed for user `30b9685e174a4523a6c36d4d0bebd5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:17.352317 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-07dcfe27-21db-4df4-8763-b13c6413f6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:17.360416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1143/2272] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:27:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:25.686592 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=63 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:27.344161 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=85 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:31.968489 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cc22a-debf-420a-a385-d007880e3814 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:31.969210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cc22a-debf-420a-a385-d007880e3814 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:31.969463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cc22a-debf-420a-a385-d007880e3814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:31.997474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d92cc22a-debf-420a-a385-d007880e3814 None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:31.998753 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d92cc22a-debf-420a-a385-d007880e3814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:32.072902 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1130/2273] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 15:27:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:32.086544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dcfe27-21db-4df4-8763-b13c6413f6f4 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:32.088228 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-07dcfe27-21db-4df4-8763-b13c6413f6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:32.170415 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dcfe27-21db-4df4-8763-b13c6413f6f4 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:32.175940 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4728a3-cc19-4bea-9cb0-b646e794a50a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:32.176581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4728a3-cc19-4bea-9cb0-b646e794a50a service swift] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:32.176857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4728a3-cc19-4bea-9cb0-b646e794a50a service swift] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:32.177121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4728a3-cc19-4bea-9cb0-b646e794a50a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:32.186562 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4728a3-cc19-4bea-9cb0-b646e794a50a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:32.188126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4728a3-cc19-4bea-9cb0-b646e794a50a service swift] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:32.196567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1144/2274] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:27:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:27:42.064694 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=25 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:42.193117 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=32 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:04.272921 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cc22a-debf-420a-a385-d007880e3814 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:04.274499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d92cc22a-debf-420a-a385-d007880e3814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:04.288778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cc22a-debf-420a-a385-d007880e3814 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:04.292730 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5554501-101a-4551-9c72-af56ab960356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:04.293496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5554501-101a-4551-9c72-af56ab960356 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:04.293818 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5554501-101a-4551-9c72-af56ab960356 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:04.294092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5554501-101a-4551-9c72-af56ab960356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:04.305295 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5554501-101a-4551-9c72-af56ab960356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:04.308952 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5554501-101a-4551-9c72-af56ab960356 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:04.322556 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1131/2275] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:28:04 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:04.588429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de151574-0635-42d3-8eb5-d90f43edf41a None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:04.589034 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de151574-0635-42d3-8eb5-d90f43edf41a None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:04.589034 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de151574-0635-42d3-8eb5-d90f43edf41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:04.608511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-de151574-0635-42d3-8eb5-d90f43edf41a None None] MFA Rules not processed for user `6f852249c66a4f8986d7a63a957ec91c`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:04.609843 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-de151574-0635-42d3-8eb5-d90f43edf41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:04.674713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1145/2276] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:28:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:08.876885 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc1ea1-89e3-4faf-bbf1-79b0c185e85e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:08.877426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc1ea1-89e3-4faf-bbf1-79b0c185e85e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:08.877426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc1ea1-89e3-4faf-bbf1-79b0c185e85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:08.898686 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-38dc1ea1-89e3-4faf-bbf1-79b0c185e85e None None] MFA Rules not processed for user `d79f95d9699b4b0393e14a9212986ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:08.899895 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-38dc1ea1-89e3-4faf-bbf1-79b0c185e85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:08.957944 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1132/2277] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:28:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:14.668353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=25 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:18.951001 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=31 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:40.651156 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59c765-4647-48b6-a335-31af86490a1d None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:40.651680 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59c765-4647-48b6-a335-31af86490a1d None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:40.651680 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59c765-4647-48b6-a335-31af86490a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:40.670865 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-ac59c765-4647-48b6-a335-31af86490a1d None None] MFA Rules not processed for user `b1afe2bda6784f9987b8aa5fa174dde2`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:40.672081 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-ac59c765-4647-48b6-a335-31af86490a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:40.680850 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1146/2278] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:28:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:43.316935 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc1ea1-89e3-4faf-bbf1-79b0c185e85e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:43.336124 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc1ea1-89e3-4faf-bbf1-79b0c185e85e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:43.343871 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e609453-0b3c-49dc-b512-2414b31a5a75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:43.344997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e609453-0b3c-49dc-b512-2414b31a5a75 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:43.345520 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e609453-0b3c-49dc-b512-2414b31a5a75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:43.345897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e609453-0b3c-49dc-b512-2414b31a5a75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:43.348328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8e609453-0b3c-49dc-b512-2414b31a5a75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:43.366529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e609453-0b3c-49dc-b512-2414b31a5a75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:43.368892 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e609453-0b3c-49dc-b512-2414b31a5a75 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:43.383439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1133/2279] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:28:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:46.283332 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59c765-4647-48b6-a335-31af86490a1d None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:46.295861 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59c765-4647-48b6-a335-31af86490a1d None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:46.299916 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe60b1-705e-4064-90c0-7ce63783b414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:46.300623 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe60b1-705e-4064-90c0-7ce63783b414 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:46.301013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe60b1-705e-4064-90c0-7ce63783b414 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:46.301349 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe60b1-705e-4064-90c0-7ce63783b414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:46.303271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-55fe60b1-705e-4064-90c0-7ce63783b414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:46.318614 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe60b1-705e-4064-90c0-7ce63783b414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:46.320095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe60b1-705e-4064-90c0-7ce63783b414 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:46.333390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1147/2280] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:28:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:48.473530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c63c2-4411-4b43-8c64-58da8b11a981 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:48.474079 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c63c2-4411-4b43-8c64-58da8b11a981 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:48.474079 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c63c2-4411-4b43-8c64-58da8b11a981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:48.495307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-361c63c2-4411-4b43-8c64-58da8b11a981 None None] MFA Rules not processed for user `c60ee64ca18e4930a14e3e17bc979ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:48.496651 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-361c63c2-4411-4b43-8c64-58da8b11a981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:48.571868 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1134/2281] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:28:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:50.055623 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e898-3953-47b1-886b-0e0a7f47b61b None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:50.056170 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e898-3953-47b1-886b-0e0a7f47b61b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:50.056170 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e898-3953-47b1-886b-0e0a7f47b61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:50.076343 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-db79e898-3953-47b1-886b-0e0a7f47b61b None None] MFA Rules not processed for user `0898fda8be3743f093dbb216f21eb931`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:50.077615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-db79e898-3953-47b1-886b-0e0a7f47b61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:50.140605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1148/2282] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:28:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:58.564006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=31 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:59.049945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c63c2-4411-4b43-8c64-58da8b11a981 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:59.069399 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c63c2-4411-4b43-8c64-58da8b11a981 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:59.077911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83247a96-8f25-4c25-95a8-07eb4f2498e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:59.078379 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83247a96-8f25-4c25-95a8-07eb4f2498e7 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:59.078722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83247a96-8f25-4c25-95a8-07eb4f2498e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:59.078983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83247a96-8f25-4c25-95a8-07eb4f2498e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:59.081401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-83247a96-8f25-4c25-95a8-07eb4f2498e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:59.094695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83247a96-8f25-4c25-95a8-07eb4f2498e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:59.096338 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83247a96-8f25-4c25-95a8-07eb4f2498e7 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:59.107256 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1135/2283] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:28:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:00.132552 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=34 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:05.768549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79e898-3953-47b1-886b-0e0a7f47b61b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:05.783562 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79e898-3953-47b1-886b-0e0a7f47b61b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:05.791703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93268f-91ad-4a23-a9bf-2a5a4a1e46f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:05.792339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93268f-91ad-4a23-a9bf-2a5a4a1e46f4 service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:05.792616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93268f-91ad-4a23-a9bf-2a5a4a1e46f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:05.792893 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93268f-91ad-4a23-a9bf-2a5a4a1e46f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:05.795031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8c93268f-91ad-4a23-a9bf-2a5a4a1e46f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:05.863322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93268f-91ad-4a23-a9bf-2a5a4a1e46f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:05.864964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93268f-91ad-4a23-a9bf-2a5a4a1e46f4 service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:05.877310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1149/2284] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:09.102351 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=6 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:09.771728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83247a96-8f25-4c25-95a8-07eb4f2498e7 service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:09.783536 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83247a96-8f25-4c25-95a8-07eb4f2498e7 service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:09.787857 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b116a-de6c-4ad1-a940-0c3fa7ebbe80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:09.788839 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b116a-de6c-4ad1-a940-0c3fa7ebbe80 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:09.789168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b116a-de6c-4ad1-a940-0c3fa7ebbe80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:09.790013 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b116a-de6c-4ad1-a940-0c3fa7ebbe80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:09.792532 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ce1b116a-de6c-4ad1-a940-0c3fa7ebbe80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:09.805596 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b116a-de6c-4ad1-a940-0c3fa7ebbe80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:09.807108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1b116a-de6c-4ad1-a940-0c3fa7ebbe80 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:09.817175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1136/2285] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:14.852771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a465c-5e20-477a-a98e-6fb607993508 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:14.853362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a465c-5e20-477a-a98e-6fb607993508 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:14.853362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a465c-5e20-477a-a98e-6fb607993508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:14.879067 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-bf2a465c-5e20-477a-a98e-6fb607993508 None None] MFA Rules not processed for user `93d86ae4501e43d59d172d2e7acfdd54`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:14.880299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-bf2a465c-5e20-477a-a98e-6fb607993508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:14.887553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1150/2286] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:15.210911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b116a-de6c-4ad1-a940-0c3fa7ebbe80 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:15.223882 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b116a-de6c-4ad1-a940-0c3fa7ebbe80 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:15.228316 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116de8fe-5618-4bcf-9fb2-40123143b1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:15.228938 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116de8fe-5618-4bcf-9fb2-40123143b1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:15.229215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116de8fe-5618-4bcf-9fb2-40123143b1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:15.229484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116de8fe-5618-4bcf-9fb2-40123143b1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:15.231675 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-116de8fe-5618-4bcf-9fb2-40123143b1b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:15.242809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116de8fe-5618-4bcf-9fb2-40123143b1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:15.244275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116de8fe-5618-4bcf-9fb2-40123143b1b0 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:15.253414 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1137/2287] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:15.864953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a465c-5e20-477a-a98e-6fb607993508 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:15.866513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-bf2a465c-5e20-477a-a98e-6fb607993508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:15.952016 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a465c-5e20-477a-a98e-6fb607993508 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:15.957925 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83038c-15c0-4be7-8982-f340e10d1d63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '098d8d7e35804ccc931ddfba07f688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:15.958492 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83038c-15c0-4be7-8982-f340e10d1d63 service placement] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:15.958768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83038c-15c0-4be7-8982-f340e10d1d63 service placement] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:15.959038 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83038c-15c0-4be7-8982-f340e10d1d63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:15.960888 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5d83038c-15c0-4be7-8982-f340e10d1d63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:15.973759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83038c-15c0-4be7-8982-f340e10d1d63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:15.975428 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83038c-15c0-4be7-8982-f340e10d1d63 service placement] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:15.985801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1151/2288] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:22.642852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116de8fe-5618-4bcf-9fb2-40123143b1b0 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:22.660677 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116de8fe-5618-4bcf-9fb2-40123143b1b0 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:22.669603 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee074cf-14c8-4ca6-bc7a-3d0bfc555120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:22.670672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee074cf-14c8-4ca6-bc7a-3d0bfc555120 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:22.671095 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee074cf-14c8-4ca6-bc7a-3d0bfc555120 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:22.672176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee074cf-14c8-4ca6-bc7a-3d0bfc555120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:22.677079 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6ee074cf-14c8-4ca6-bc7a-3d0bfc555120 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:22.697265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee074cf-14c8-4ca6-bc7a-3d0bfc555120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:22.714468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee074cf-14c8-4ca6-bc7a-3d0bfc555120 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:22.723564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1138/2289] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:25.981587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=63 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:26.650890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83038c-15c0-4be7-8982-f340e10d1d63 service placement] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:26.670504 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83038c-15c0-4be7-8982-f340e10d1d63 service placement] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:26.677977 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442f052-113b-466c-9a59-33a2db84159b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:26.678613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442f052-113b-466c-9a59-33a2db84159b service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:26.678893 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442f052-113b-466c-9a59-33a2db84159b service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:26.679167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442f052-113b-466c-9a59-33a2db84159b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:26.681335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-7442f052-113b-466c-9a59-33a2db84159b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:26.695566 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442f052-113b-466c-9a59-33a2db84159b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:26.698112 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442f052-113b-466c-9a59-33a2db84159b service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:26.714381 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1152/2290] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:27.079287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee074cf-14c8-4ca6-bc7a-3d0bfc555120 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:27.095198 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee074cf-14c8-4ca6-bc7a-3d0bfc555120 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:27.101623 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80080dd2-49bf-4082-ba72-46497e809aef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:27.102271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80080dd2-49bf-4082-ba72-46497e809aef service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:27.102536 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80080dd2-49bf-4082-ba72-46497e809aef service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:27.102962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80080dd2-49bf-4082-ba72-46497e809aef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:27.104931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-80080dd2-49bf-4082-ba72-46497e809aef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:27.119582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80080dd2-49bf-4082-ba72-46497e809aef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:27.121453 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80080dd2-49bf-4082-ba72-46497e809aef service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:27.134698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1139/2291] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:31.896432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442f052-113b-466c-9a59-33a2db84159b service nova] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:31.914714 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442f052-113b-466c-9a59-33a2db84159b service nova] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:31.921218 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05e64e-ac83-487f-95b3-b28e88d319c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:31.922263 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05e64e-ac83-487f-95b3-b28e88d319c5 service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:31.922539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05e64e-ac83-487f-95b3-b28e88d319c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:31.922813 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05e64e-ac83-487f-95b3-b28e88d319c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:31.924832 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-fc05e64e-ac83-487f-95b3-b28e88d319c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:31.942281 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05e64e-ac83-487f-95b3-b28e88d319c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:31.944196 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05e64e-ac83-487f-95b3-b28e88d319c5 service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:31.961959 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1153/2292] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:32.369173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80080dd2-49bf-4082-ba72-46497e809aef service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:32.370719 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-80080dd2-49bf-4082-ba72-46497e809aef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:32.386368 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80080dd2-49bf-4082-ba72-46497e809aef service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:32.392385 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e5a48-69c5-4699-8cb3-e2ca9a76232e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:32.393082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e5a48-69c5-4699-8cb3-e2ca9a76232e service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:32.393741 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e5a48-69c5-4699-8cb3-e2ca9a76232e service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:32.394028 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543e5a48-69c5-4699-8cb3-e2ca9a76232e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:32.395998 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-543e5a48-69c5-4699-8cb3-e2ca9a76232e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:32.409828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e5a48-69c5-4699-8cb3-e2ca9a76232e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:32.411437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543e5a48-69c5-4699-8cb3-e2ca9a76232e service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:32.423199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1140/2293] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:41.450859 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05e64e-ac83-487f-95b3-b28e88d319c5 service nova] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:41.460770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05e64e-ac83-487f-95b3-b28e88d319c5 service nova] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:41.464484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280458c1-6c0b-4382-abfa-57e166331ead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:41.465104 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280458c1-6c0b-4382-abfa-57e166331ead service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:41.465384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280458c1-6c0b-4382-abfa-57e166331ead service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:41.465865 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280458c1-6c0b-4382-abfa-57e166331ead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:41.467692 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-280458c1-6c0b-4382-abfa-57e166331ead service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:41.480301 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280458c1-6c0b-4382-abfa-57e166331ead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:41.481706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280458c1-6c0b-4382-abfa-57e166331ead service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:41.492013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1154/2294] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:29:42.417854 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=30 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:44.858235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e5a48-69c5-4699-8cb3-e2ca9a76232e service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:44.859711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-543e5a48-69c5-4699-8cb3-e2ca9a76232e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:44.874213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543e5a48-69c5-4699-8cb3-e2ca9a76232e service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:44.879055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efa88e-275b-47de-9122-312b4475750f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:44.879836 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efa88e-275b-47de-9122-312b4475750f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:44.880116 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efa88e-275b-47de-9122-312b4475750f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:44.880390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efa88e-275b-47de-9122-312b4475750f admin admin] PATH_INFO: `/v3/users/6f852249c66a4f8986d7a63a957ec91c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:44.882094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efa88e-275b-47de-9122-312b4475750f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f852249c66a4f8986d7a63a957ec91c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:44.884511 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efa88e-275b-47de-9122-312b4475750f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:44.937173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-15efa88e-275b-47de-9122-312b4475750f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f852249c66a4f8986d7a63a957ec91c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:44.938101 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-15efa88e-275b-47de-9122-312b4475750f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f852249c66a4f8986d7a63a957ec91c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:44.953235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-15efa88e-275b-47de-9122-312b4475750f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f852249c66a4f8986d7a63a957ec91c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:44.961560 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-15efa88e-275b-47de-9122-312b4475750f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f852249c66a4f8986d7a63a957ec91c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:44.969018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-15efa88e-275b-47de-9122-312b4475750f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f852249c66a4f8986d7a63a957ec91c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:44.975827 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1141/2295] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:29:44 2022] DELETE /identity/v3/users/6f852249c66a4f8986d7a63a957ec91c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:45.163999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280458c1-6c0b-4382-abfa-57e166331ead service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:45.175671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280458c1-6c0b-4382-abfa-57e166331ead service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:45.244630 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d93548-019f-49b6-b7ec-ccde4cc6214a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:45.245402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d93548-019f-49b6-b7ec-ccde4cc6214a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:45.245695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d93548-019f-49b6-b7ec-ccde4cc6214a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:45.245999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d93548-019f-49b6-b7ec-ccde4cc6214a admin admin] PATH_INFO: `/v3/projects/a359662d2a444976af8b1b9cc351edb9` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:45.247255 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d93548-019f-49b6-b7ec-ccde4cc6214a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a359662d2a444976af8b1b9cc351edb9)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:45.249201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d93548-019f-49b6-b7ec-ccde4cc6214a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:45.301220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-97d93548-019f-49b6-b7ec-ccde4cc6214a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a359662d2a444976af8b1b9cc351edb9', 'request_id': 'req-97d93548-019f-49b6-b7ec-ccde4cc6214a'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:45.310310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-97d93548-019f-49b6-b7ec-ccde4cc6214a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a359662d2a444976af8b1b9cc351edb9', 'request_id': 'req-97d93548-019f-49b6-b7ec-ccde4cc6214a'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:45.317518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-97d93548-019f-49b6-b7ec-ccde4cc6214a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a359662d2a444976af8b1b9cc351edb9', 'request_id': 'req-97d93548-019f-49b6-b7ec-ccde4cc6214a'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:45.329040 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-97d93548-019f-49b6-b7ec-ccde4cc6214a admin admin] The token cache is being invalidate because project a359662d2a444976af8b1b9cc351edb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:29:45.329370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-97d93548-019f-49b6-b7ec-ccde4cc6214a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:45.330087 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-97d93548-019f-49b6-b7ec-ccde4cc6214a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:45.332026 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1155/2296] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:29:45 2022] DELETE /identity/v3/projects/a359662d2a444976af8b1b9cc351edb9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:45.337679 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580f1b6-6c3f-4dea-ad9e-80fd75e83c4d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:45.337997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580f1b6-6c3f-4dea-ad9e-80fd75e83c4d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:45.338277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580f1b6-6c3f-4dea-ad9e-80fd75e83c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:45.361285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-0580f1b6-6c3f-4dea-ad9e-80fd75e83c4d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:45.362574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0580f1b6-6c3f-4dea-ad9e-80fd75e83c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:45.426864 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1142/2297] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:45.436099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800e2fd-1644-407e-a21b-7ba7e87d2fbf None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:45.436411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800e2fd-1644-407e-a21b-7ba7e87d2fbf None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:45.436678 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800e2fd-1644-407e-a21b-7ba7e87d2fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:45.463183 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-7800e2fd-1644-407e-a21b-7ba7e87d2fbf None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:45.465049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-7800e2fd-1644-407e-a21b-7ba7e87d2fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:45.474984 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1156/2298] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:45.479721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580f1b6-6c3f-4dea-ad9e-80fd75e83c4d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:45.490530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580f1b6-6c3f-4dea-ad9e-80fd75e83c4d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:45.495478 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad9fcf-db33-4107-851d-27b01fb7fc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:45.496113 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad9fcf-db33-4107-851d-27b01fb7fc85 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:45.496500 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad9fcf-db33-4107-851d-27b01fb7fc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:45.496781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad9fcf-db33-4107-851d-27b01fb7fc85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:45.497491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad9fcf-db33-4107-851d-27b01fb7fc85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:29:45.497808 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad9fcf-db33-4107-851d-27b01fb7fc85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:45.499449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad9fcf-db33-4107-851d-27b01fb7fc85 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:45.512069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1143/2299] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:29:45.516920 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800e2fd-1644-407e-a21b-7ba7e87d2fbf None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:45.527119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7800e2fd-1644-407e-a21b-7ba7e87d2fbf None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:45.532003 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b59e4e-c253-4bc5-8ff6-a19acf036a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:45.532691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b59e4e-c253-4bc5-8ff6-a19acf036a49 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:45.532961 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b59e4e-c253-4bc5-8ff6-a19acf036a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:45.533235 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b59e4e-c253-4bc5-8ff6-a19acf036a49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:45.533907 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b59e4e-c253-4bc5-8ff6-a19acf036a49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:45.535891 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b59e4e-c253-4bc5-8ff6-a19acf036a49 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:45.569857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1157/2300] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:29:45 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:29:45.571012 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad9fcf-db33-4107-851d-27b01fb7fc85 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:45.584974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad9fcf-db33-4107-851d-27b01fb7fc85 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:45.654033 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689dd755-c612-4244-907d-fd0246e63cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:45.654673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dd755-c612-4244-907d-fd0246e63cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:45.654951 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dd755-c612-4244-907d-fd0246e63cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:45.655229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689dd755-c612-4244-907d-fd0246e63cb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:45.656041 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689dd755-c612-4244-907d-fd0246e63cb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:45.658334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689dd755-c612-4244-907d-fd0246e63cb0 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:45.670445 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:45.705810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1144/2301] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:29:45 2022] POST /identity/v3/users => generated 303 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:29:45.717318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b59e4e-c253-4bc5-8ff6-a19acf036a49 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:45.730627 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b59e4e-c253-4bc5-8ff6-a19acf036a49 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:45.740224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004a79d-5269-4f63-b54f-73386f106c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:45.740887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004a79d-5269-4f63-b54f-73386f106c08 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:45.742725 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004a79d-5269-4f63-b54f-73386f106c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:45.742725 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004a79d-5269-4f63-b54f-73386f106c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:45.742725 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004a79d-5269-4f63-b54f-73386f106c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:45.744743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004a79d-5269-4f63-b54f-73386f106c08 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:45.759536 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1158/2302] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:29:45.763601 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689dd755-c612-4244-907d-fd0246e63cb0 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:45.774307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689dd755-c612-4244-907d-fd0246e63cb0 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:45.779090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2b9e1-4e53-4f64-a9db-97db52036653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:45.779757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2b9e1-4e53-4f64-a9db-97db52036653 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:45.780038 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2b9e1-4e53-4f64-a9db-97db52036653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:45.780352 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2b9e1-4e53-4f64-a9db-97db52036653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:45.781053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2b9e1-4e53-4f64-a9db-97db52036653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:45.782750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2b9e1-4e53-4f64-a9db-97db52036653 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:45.794111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1145/2303] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:29:45.798388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004a79d-5269-4f63-b54f-73386f106c08 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:45.809943 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004a79d-5269-4f63-b54f-73386f106c08 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:45.818432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88efe4-1612-4c40-8339-e550147b6ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:45.819118 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88efe4-1612-4c40-8339-e550147b6ec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:45.819397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88efe4-1612-4c40-8339-e550147b6ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:45.819665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88efe4-1612-4c40-8339-e550147b6ec0 admin admin] PATH_INFO: `/v3/projects/2ae6f02c5c7b44cbb4ef05a554515ec0/users/3967136fb78f4575b9f3edc99bac3f23/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:45.831163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:45.841417 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88efe4-1612-4c40-8339-e550147b6ec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae6f02c5c7b44cbb4ef05a554515ec0, user_id=3967136fb78f4575b9f3edc99bac3f23, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:45.845856 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88efe4-1612-4c40-8339-e550147b6ec0 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:45.870462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1159/2304] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:29:45 2022] PUT /identity/v3/projects/2ae6f02c5c7b44cbb4ef05a554515ec0/users/3967136fb78f4575b9f3edc99bac3f23/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:29:45.875538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d52b60-7b0b-4980-95e0-8fe889f84e6d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:45.876021 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d52b60-7b0b-4980-95e0-8fe889f84e6d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:45.876134 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d52b60-7b0b-4980-95e0-8fe889f84e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:45.902958 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-34d52b60-7b0b-4980-95e0-8fe889f84e6d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:45.904285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-34d52b60-7b0b-4980-95e0-8fe889f84e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:45.987297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1146/2305] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:29:45.995754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88efe4-1612-4c40-8339-e550147b6ec0 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:45.998093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9a88efe4-1612-4c40-8339-e550147b6ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:46.114226 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88efe4-1612-4c40-8339-e550147b6ec0 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:46.120062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab478383-3813-472c-a08c-20d144bee574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:46.121247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab478383-3813-472c-a08c-20d144bee574 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:46.121652 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab478383-3813-472c-a08c-20d144bee574 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:46.121943 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab478383-3813-472c-a08c-20d144bee574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:46.134605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab478383-3813-472c-a08c-20d144bee574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:46.136031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab478383-3813-472c-a08c-20d144bee574 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:46.149134 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1160/2306] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:46.870438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a0a8a-0e31-498a-aa0e-8f0e28ce417e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:46.870717 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a0a8a-0e31-498a-aa0e-8f0e28ce417e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:46.871193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a0a8a-0e31-498a-aa0e-8f0e28ce417e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:46.912602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e44a0a8a-0e31-498a-aa0e-8f0e28ce417e None None] MFA Rules not processed for user `3967136fb78f4575b9f3edc99bac3f23`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:46.914773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e44a0a8a-0e31-498a-aa0e-8f0e28ce417e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:47.022105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1147/2307] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:46 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:47.026289 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab478383-3813-472c-a08c-20d144bee574 service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:47.037982 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab478383-3813-472c-a08c-20d144bee574 service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:47.043427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e913ce-2617-4724-834d-f8ca6912b4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:47.044172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e913ce-2617-4724-834d-f8ca6912b4a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:47.044575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e913ce-2617-4724-834d-f8ca6912b4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:47.044950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e913ce-2617-4724-834d-f8ca6912b4a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:47.046027 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e913ce-2617-4724-834d-f8ca6912b4a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:47.048494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e913ce-2617-4724-834d-f8ca6912b4a8 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:47.097854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1161/2308] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:29:47 2022] POST /identity/v3/projects => generated 403 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:29:47.100968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a0a8a-0e31-498a-aa0e-8f0e28ce417e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:47.119236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a0a8a-0e31-498a-aa0e-8f0e28ce417e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:47.217690 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244963a-ada6-4cc6-a032-bf53601280a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:47.218480 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244963a-ada6-4cc6-a032-bf53601280a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:47.218822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244963a-ada6-4cc6-a032-bf53601280a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:47.219162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244963a-ada6-4cc6-a032-bf53601280a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:47.219901 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244963a-ada6-4cc6-a032-bf53601280a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:47.222013 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244963a-ada6-4cc6-a032-bf53601280a7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:47.239713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:47.246397 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1148/2309] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:29:47 2022] POST /identity/v3/users => generated 308 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:29:47.250494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e913ce-2617-4724-834d-f8ca6912b4a8 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:47.261945 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e913ce-2617-4724-834d-f8ca6912b4a8 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:47.266942 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafc3e5-09e6-459c-a1d1-85a6c2370966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:47.267831 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafc3e5-09e6-459c-a1d1-85a6c2370966 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:47.268110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafc3e5-09e6-459c-a1d1-85a6c2370966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:47.268550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafc3e5-09e6-459c-a1d1-85a6c2370966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:47.269464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafc3e5-09e6-459c-a1d1-85a6c2370966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:47.271237 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafc3e5-09e6-459c-a1d1-85a6c2370966 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:47.282578 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1162/2310] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:29:47.287638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244963a-ada6-4cc6-a032-bf53601280a7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:47.301100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244963a-ada6-4cc6-a032-bf53601280a7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:47.313422 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dfb86-645f-4c64-b4ca-1b10f708aecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:47.314496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dfb86-645f-4c64-b4ca-1b10f708aecd admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:47.314890 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dfb86-645f-4c64-b4ca-1b10f708aecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:47.315275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dfb86-645f-4c64-b4ca-1b10f708aecd admin admin] PATH_INFO: `/v3/projects/f4d0bea4a4c74eb3b140b529eb870c77/users/8eb4c0aa6d3245c5b1cdd454175d636f/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:47.337525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742dfb86-645f-4c64-b4ca-1b10f708aecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d0bea4a4c74eb3b140b529eb870c77, user_id=8eb4c0aa6d3245c5b1cdd454175d636f, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:47.340429 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742dfb86-645f-4c64-b4ca-1b10f708aecd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:47.360119 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1149/2311] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:29:47 2022] PUT /identity/v3/projects/f4d0bea4a4c74eb3b140b529eb870c77/users/8eb4c0aa6d3245c5b1cdd454175d636f/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:47.364353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafc3e5-09e6-459c-a1d1-85a6c2370966 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:47.375533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafc3e5-09e6-459c-a1d1-85a6c2370966 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:47.447163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179910e-a64d-4082-bc8f-c13af6cf4939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:47.448234 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179910e-a64d-4082-bc8f-c13af6cf4939 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:47.448851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179910e-a64d-4082-bc8f-c13af6cf4939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:47.449295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179910e-a64d-4082-bc8f-c13af6cf4939 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:47.450359 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179910e-a64d-4082-bc8f-c13af6cf4939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:47.453623 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179910e-a64d-4082-bc8f-c13af6cf4939 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:47.465657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1163/2312] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:29:47.470054 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dfb86-645f-4c64-b4ca-1b10f708aecd admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:47.482475 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dfb86-645f-4c64-b4ca-1b10f708aecd admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:47.490992 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acec73f-408f-4b72-8378-c3cee8d4e500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:47.491759 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acec73f-408f-4b72-8378-c3cee8d4e500 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:47.492525 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acec73f-408f-4b72-8378-c3cee8d4e500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:47.492809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acec73f-408f-4b72-8378-c3cee8d4e500 admin admin] PATH_INFO: `/v3/projects/f4d0bea4a4c74eb3b140b529eb870c77/users/8eb4c0aa6d3245c5b1cdd454175d636f/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:47.495593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acec73f-408f-4b72-8378-c3cee8d4e500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d0bea4a4c74eb3b140b529eb870c77, user_id=8eb4c0aa6d3245c5b1cdd454175d636f, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:47.498450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acec73f-408f-4b72-8378-c3cee8d4e500 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:47.515507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1150/2313] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:29:47 2022] PUT /identity/v3/projects/f4d0bea4a4c74eb3b140b529eb870c77/users/8eb4c0aa6d3245c5b1cdd454175d636f/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:48.244107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bec124-1459-4837-ba9b-783d5f7d38e7 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:48.244587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bec124-1459-4837-ba9b-783d5f7d38e7 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:48.244709 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bec124-1459-4837-ba9b-783d5f7d38e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:48.283352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-99bec124-1459-4837-ba9b-783d5f7d38e7 None None] MFA Rules not processed for user `8eb4c0aa6d3245c5b1cdd454175d636f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:48.285415 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-99bec124-1459-4837-ba9b-783d5f7d38e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:48.366919 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:48.445107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1164/2314] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:48 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:48.870221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a160cfa-e248-428d-8222-81fbcb907b94 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:48.872059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a160cfa-e248-428d-8222-81fbcb907b94 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:48.872547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a160cfa-e248-428d-8222-81fbcb907b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:48.899387 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-9a160cfa-e248-428d-8222-81fbcb907b94 None None] MFA Rules not processed for user `d79f95d9699b4b0393e14a9212986ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:48.900609 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9a160cfa-e248-428d-8222-81fbcb907b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:48.963140 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1151/2315] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:49.409680 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bec124-1459-4837-ba9b-783d5f7d38e7 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:49.411331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-99bec124-1459-4837-ba9b-783d5f7d38e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:49.474029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bec124-1459-4837-ba9b-783d5f7d38e7 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:49.477644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155f550-f199-44d4-a4fa-25a8e613b409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:49.478309 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155f550-f199-44d4-a4fa-25a8e613b409 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:49.478585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155f550-f199-44d4-a4fa-25a8e613b409 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:49.478855 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155f550-f199-44d4-a4fa-25a8e613b409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:49.480690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-0155f550-f199-44d4-a4fa-25a8e613b409 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:49.490464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155f550-f199-44d4-a4fa-25a8e613b409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:49.491905 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155f550-f199-44d4-a4fa-25a8e613b409 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:49.500220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1165/2316] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:49.907001 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a160cfa-e248-428d-8222-81fbcb907b94 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:49.988922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a160cfa-e248-428d-8222-81fbcb907b94 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:50.000535 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed0e3e-8ca9-43ee-a922-3349e63f9731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:50.001230 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed0e3e-8ca9-43ee-a922-3349e63f9731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:50.001608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed0e3e-8ca9-43ee-a922-3349e63f9731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:50.001757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed0e3e-8ca9-43ee-a922-3349e63f9731 admin admin] PATH_INFO: `/v3/users/3967136fb78f4575b9f3edc99bac3f23` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:50.002977 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed0e3e-8ca9-43ee-a922-3349e63f9731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3967136fb78f4575b9f3edc99bac3f23)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:50.004948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed0e3e-8ca9-43ee-a922-3349e63f9731 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:50.056577 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-88ed0e3e-8ca9-43ee-a922-3349e63f9731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3967136fb78f4575b9f3edc99bac3f23'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:50.057372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-88ed0e3e-8ca9-43ee-a922-3349e63f9731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3967136fb78f4575b9f3edc99bac3f23'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:50.070227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-88ed0e3e-8ca9-43ee-a922-3349e63f9731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3967136fb78f4575b9f3edc99bac3f23'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:50.072772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:50.079831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-88ed0e3e-8ca9-43ee-a922-3349e63f9731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3967136fb78f4575b9f3edc99bac3f23'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:50.089886 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-88ed0e3e-8ca9-43ee-a922-3349e63f9731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3967136fb78f4575b9f3edc99bac3f23'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:50.097996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1152/2317] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:29:49 2022] DELETE /identity/v3/users/3967136fb78f4575b9f3edc99bac3f23 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:50.286991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155f550-f199-44d4-a4fa-25a8e613b409 service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:50.299066 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155f550-f199-44d4-a4fa-25a8e613b409 service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:50.355931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31936aea-8beb-46f4-a54d-c4133f4f702e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:50.356575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31936aea-8beb-46f4-a54d-c4133f4f702e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:50.356846 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31936aea-8beb-46f4-a54d-c4133f4f702e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:50.357115 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31936aea-8beb-46f4-a54d-c4133f4f702e admin admin] PATH_INFO: `/v3/projects/2ae6f02c5c7b44cbb4ef05a554515ec0` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:50.358362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31936aea-8beb-46f4-a54d-c4133f4f702e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ae6f02c5c7b44cbb4ef05a554515ec0)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:50.360286 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31936aea-8beb-46f4-a54d-c4133f4f702e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:50.407552 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-31936aea-8beb-46f4-a54d-c4133f4f702e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ae6f02c5c7b44cbb4ef05a554515ec0', 'request_id': 'req-31936aea-8beb-46f4-a54d-c4133f4f702e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:50.415602 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-31936aea-8beb-46f4-a54d-c4133f4f702e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ae6f02c5c7b44cbb4ef05a554515ec0', 'request_id': 'req-31936aea-8beb-46f4-a54d-c4133f4f702e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:50.423195 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-31936aea-8beb-46f4-a54d-c4133f4f702e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ae6f02c5c7b44cbb4ef05a554515ec0', 'request_id': 'req-31936aea-8beb-46f4-a54d-c4133f4f702e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:50.438108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-31936aea-8beb-46f4-a54d-c4133f4f702e admin admin] The token cache is being invalidate because project 2ae6f02c5c7b44cbb4ef05a554515ec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:29:50.438437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-31936aea-8beb-46f4-a54d-c4133f4f702e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:50.439555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-31936aea-8beb-46f4-a54d-c4133f4f702e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:50.442131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1166/2318] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:29:50 2022] DELETE /identity/v3/projects/2ae6f02c5c7b44cbb4ef05a554515ec0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:50.446098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed0e3e-8ca9-43ee-a922-3349e63f9731 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:50.448042 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-88ed0e3e-8ca9-43ee-a922-3349e63f9731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:50.463102 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed0e3e-8ca9-43ee-a922-3349e63f9731 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:50.532507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e6ed6-cad4-4c50-8891-70ed191228bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:50.533300 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e6ed6-cad4-4c50-8891-70ed191228bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:50.533579 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e6ed6-cad4-4c50-8891-70ed191228bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:50.533888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e6ed6-cad4-4c50-8891-70ed191228bb admin admin] PATH_INFO: `/v3/users/8eb4c0aa6d3245c5b1cdd454175d636f` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:50.535207 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e6ed6-cad4-4c50-8891-70ed191228bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eb4c0aa6d3245c5b1cdd454175d636f)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:50.537313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e6ed6-cad4-4c50-8891-70ed191228bb admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:50.588169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-080e6ed6-cad4-4c50-8891-70ed191228bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eb4c0aa6d3245c5b1cdd454175d636f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:50.589146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-080e6ed6-cad4-4c50-8891-70ed191228bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eb4c0aa6d3245c5b1cdd454175d636f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:50.604715 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-080e6ed6-cad4-4c50-8891-70ed191228bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eb4c0aa6d3245c5b1cdd454175d636f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:50.613666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-080e6ed6-cad4-4c50-8891-70ed191228bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eb4c0aa6d3245c5b1cdd454175d636f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:50.621825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-080e6ed6-cad4-4c50-8891-70ed191228bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eb4c0aa6d3245c5b1cdd454175d636f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:50.627968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1153/2319] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:29:50 2022] DELETE /identity/v3/users/8eb4c0aa6d3245c5b1cdd454175d636f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:50.818019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31936aea-8beb-46f4-a54d-c4133f4f702e admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:50.820029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-31936aea-8beb-46f4-a54d-c4133f4f702e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:50.836369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31936aea-8beb-46f4-a54d-c4133f4f702e admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:50.855654 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:50.923818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b01a1-4466-4539-ab14-0ede176aeb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:50.924612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b01a1-4466-4539-ab14-0ede176aeb18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:50.924993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b01a1-4466-4539-ab14-0ede176aeb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:50.925366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b01a1-4466-4539-ab14-0ede176aeb18 admin admin] PATH_INFO: `/v3/projects/f4d0bea4a4c74eb3b140b529eb870c77` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:50.926931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b01a1-4466-4539-ab14-0ede176aeb18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4d0bea4a4c74eb3b140b529eb870c77)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:50.929524 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b01a1-4466-4539-ab14-0ede176aeb18 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:51.019998 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-587b01a1-4466-4539-ab14-0ede176aeb18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4d0bea4a4c74eb3b140b529eb870c77', 'request_id': 'req-587b01a1-4466-4539-ab14-0ede176aeb18'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:51.034284 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-587b01a1-4466-4539-ab14-0ede176aeb18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4d0bea4a4c74eb3b140b529eb870c77', 'request_id': 'req-587b01a1-4466-4539-ab14-0ede176aeb18'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:51.045568 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-587b01a1-4466-4539-ab14-0ede176aeb18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4d0bea4a4c74eb3b140b529eb870c77', 'request_id': 'req-587b01a1-4466-4539-ab14-0ede176aeb18'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:51.058675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-587b01a1-4466-4539-ab14-0ede176aeb18 admin admin] The token cache is being invalidate because project f4d0bea4a4c74eb3b140b529eb870c77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:29:51.059141 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-587b01a1-4466-4539-ab14-0ede176aeb18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:51.059879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-587b01a1-4466-4539-ab14-0ede176aeb18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:51.064712 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1167/2320] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:29:50 2022] DELETE /identity/v3/projects/f4d0bea4a4c74eb3b140b529eb870c77 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:51.066469 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d419be-e33b-48c9-b677-ee9608575290 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:51.066948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d419be-e33b-48c9-b677-ee9608575290 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:51.067156 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d419be-e33b-48c9-b677-ee9608575290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:51.090045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-33d419be-e33b-48c9-b677-ee9608575290 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:51.090238 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-33d419be-e33b-48c9-b677-ee9608575290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:51.151283 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1154/2321] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:51.159185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043532e-dd88-4d90-a2e6-8f8537f332aa None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:51.159465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043532e-dd88-4d90-a2e6-8f8537f332aa None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:51.159740 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043532e-dd88-4d90-a2e6-8f8537f332aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:51.179632 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-9043532e-dd88-4d90-a2e6-8f8537f332aa None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:51.180838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9043532e-dd88-4d90-a2e6-8f8537f332aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:51.188783 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1168/2322] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:51.193313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d419be-e33b-48c9-b677-ee9608575290 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:51.199928 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:51.204147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d419be-e33b-48c9-b677-ee9608575290 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:51.211654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233fee0-4319-402a-9621-de8673feac55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:51.213031 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233fee0-4319-402a-9621-de8673feac55 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:51.213272 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233fee0-4319-402a-9621-de8673feac55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:51.213804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233fee0-4319-402a-9621-de8673feac55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:51.214531 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233fee0-4319-402a-9621-de8673feac55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:29:51.214833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233fee0-4319-402a-9621-de8673feac55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:51.216751 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233fee0-4319-402a-9621-de8673feac55 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:51.230053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1155/2323] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:29:51.234023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043532e-dd88-4d90-a2e6-8f8537f332aa None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:51.243605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043532e-dd88-4d90-a2e6-8f8537f332aa None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:51.248114 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b21ba-6438-4590-b2eb-e037f50c451a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:51.248734 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b21ba-6438-4590-b2eb-e037f50c451a admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:51.249008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b21ba-6438-4590-b2eb-e037f50c451a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:51.249278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b21ba-6438-4590-b2eb-e037f50c451a admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:51.249976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b21ba-6438-4590-b2eb-e037f50c451a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:51.251883 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b21ba-6438-4590-b2eb-e037f50c451a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:51.281006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1169/2324] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:29:51 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:29:51.285457 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233fee0-4319-402a-9621-de8673feac55 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:51.296477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233fee0-4319-402a-9621-de8673feac55 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:51.309669 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b21ba-6438-4590-b2eb-e037f50c451a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:51.312004 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-732b21ba-6438-4590-b2eb-e037f50c451a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:51.377872 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e76bc-6667-4d7e-8331-e3b0e13b92bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:51.378568 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e76bc-6667-4d7e-8331-e3b0e13b92bd admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:51.378904 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e76bc-6667-4d7e-8331-e3b0e13b92bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:51.379179 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e76bc-6667-4d7e-8331-e3b0e13b92bd admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:51.379860 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e76bc-6667-4d7e-8331-e3b0e13b92bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:51.381933 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e76bc-6667-4d7e-8331-e3b0e13b92bd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:51.402224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b21ba-6438-4590-b2eb-e037f50c451a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:51.409562 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba0911-e6a9-4593-b172-d38730c63fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:51.410550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba0911-e6a9-4593-b172-d38730c63fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:51.410931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba0911-e6a9-4593-b172-d38730c63fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:51.411310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba0911-e6a9-4593-b172-d38730c63fca admin admin] PATH_INFO: `/v3/users/b1afe2bda6784f9987b8aa5fa174dde2` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:51.412653 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba0911-e6a9-4593-b172-d38730c63fca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1afe2bda6784f9987b8aa5fa174dde2)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:51.414415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1156/2325] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:29:51 2022] POST /identity/v3/users => generated 282 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:29:51.415147 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba0911-e6a9-4593-b172-d38730c63fca admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:51.419665 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e76bc-6667-4d7e-8331-e3b0e13b92bd admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:51.431170 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e76bc-6667-4d7e-8331-e3b0e13b92bd admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:51.433353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:51.436581 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36881c7a-e6fd-4df5-832e-8832beeeec62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:51.437217 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36881c7a-e6fd-4df5-832e-8832beeeec62 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:51.437543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36881c7a-e6fd-4df5-832e-8832beeeec62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:51.437827 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36881c7a-e6fd-4df5-832e-8832beeeec62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:51.438620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36881c7a-e6fd-4df5-832e-8832beeeec62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:51.440594 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36881c7a-e6fd-4df5-832e-8832beeeec62 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:51.454870 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1157/2326] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:29:51.459631 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36881c7a-e6fd-4df5-832e-8832beeeec62 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:51.470627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36881c7a-e6fd-4df5-832e-8832beeeec62 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:51.476519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8aaa2-998c-4aff-a65e-388ab84b5c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:51.477215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8aaa2-998c-4aff-a65e-388ab84b5c40 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:51.477507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8aaa2-998c-4aff-a65e-388ab84b5c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:51.477827 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8aaa2-998c-4aff-a65e-388ab84b5c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:51.478538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8aaa2-998c-4aff-a65e-388ab84b5c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:51.480860 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8aaa2-998c-4aff-a65e-388ab84b5c40 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:51.491946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-75ba0911-e6a9-4593-b172-d38730c63fca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1afe2bda6784f9987b8aa5fa174dde2'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:51.492725 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1158/2327] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:29:51.496948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8aaa2-998c-4aff-a65e-388ab84b5c40 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:51.502309 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-75ba0911-e6a9-4593-b172-d38730c63fca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1afe2bda6784f9987b8aa5fa174dde2'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:51.504754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-75ba0911-e6a9-4593-b172-d38730c63fca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1afe2bda6784f9987b8aa5fa174dde2'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:51.508847 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8aaa2-998c-4aff-a65e-388ab84b5c40 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:51.514298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-75ba0911-e6a9-4593-b172-d38730c63fca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1afe2bda6784f9987b8aa5fa174dde2'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:51.518709 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8399c-9ca0-49a0-8cb8-c99ab35d5bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:51.520537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8399c-9ca0-49a0-8cb8-c99ab35d5bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:51.521023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8399c-9ca0-49a0-8cb8-c99ab35d5bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:51.521566 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8399c-9ca0-49a0-8cb8-c99ab35d5bbe admin admin] PATH_INFO: `/v3/projects/7dea182791a346868d0d7382171a3d4d/users/63f17dc927214bf888d6fc735158a49a/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:51.534043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-75ba0911-e6a9-4593-b172-d38730c63fca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1afe2bda6784f9987b8aa5fa174dde2'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:51.544638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1170/2328] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:29:51 2022] DELETE /identity/v3/users/b1afe2bda6784f9987b8aa5fa174dde2 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:51.564302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8399c-9ca0-49a0-8cb8-c99ab35d5bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dea182791a346868d0d7382171a3d4d, user_id=63f17dc927214bf888d6fc735158a49a, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:51.571942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa8399c-9ca0-49a0-8cb8-c99ab35d5bbe admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:51.599771 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1159/2329] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:29:51 2022] PUT /identity/v3/projects/7dea182791a346868d0d7382171a3d4d/users/63f17dc927214bf888d6fc735158a49a/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:29:51.611132 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703fb42-3f31-474f-8da3-bed8e6aee471 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:51.611417 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703fb42-3f31-474f-8da3-bed8e6aee471 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:51.611693 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703fb42-3f31-474f-8da3-bed8e6aee471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:51.641357 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-4703fb42-3f31-474f-8da3-bed8e6aee471 None None] MFA Rules not processed for user `63f17dc927214bf888d6fc735158a49a`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:51.642605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4703fb42-3f31-474f-8da3-bed8e6aee471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:51.710372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1171/2330] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:51 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:51.715035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8399c-9ca0-49a0-8cb8-c99ab35d5bbe admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:51.716902 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-daa8399c-9ca0-49a0-8cb8-c99ab35d5bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:51.733825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703fb42-3f31-474f-8da3-bed8e6aee471 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:51.735400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4703fb42-3f31-474f-8da3-bed8e6aee471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:51.814259 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8399c-9ca0-49a0-8cb8-c99ab35d5bbe admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:51.815942 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703fb42-3f31-474f-8da3-bed8e6aee471 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:51.819984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae877bcf-16d7-4770-a15c-66a13affa883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:51.820496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae877bcf-16d7-4770-a15c-66a13affa883 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:51.820802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae877bcf-16d7-4770-a15c-66a13affa883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:51.821195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae877bcf-16d7-4770-a15c-66a13affa883 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:51.822104 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae877bcf-16d7-4770-a15c-66a13affa883 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:51.822656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c5c84-928f-46b2-aa21-1a69c460146e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:51.823377 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c5c84-928f-46b2-aa21-1a69c460146e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:51.823718 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c5c84-928f-46b2-aa21-1a69c460146e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:51.824053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0c5c84-928f-46b2-aa21-1a69c460146e admin admin] PATH_INFO: `/v3/projects/dc512d8e2ab848fda424f9eeb6c816c9` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:51.824951 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae877bcf-16d7-4770-a15c-66a13affa883 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:51.825484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c5c84-928f-46b2-aa21-1a69c460146e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc512d8e2ab848fda424f9eeb6c816c9)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:51.827684 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0c5c84-928f-46b2-aa21-1a69c460146e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:51.866971 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1160/2331] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:29:51 2022] POST /identity/v3/projects => generated 363 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:29:51.866971 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae877bcf-16d7-4770-a15c-66a13affa883 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:51.870940 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:51.901812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:51.904775 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4c0c5c84-928f-46b2-aa21-1a69c460146e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc512d8e2ab848fda424f9eeb6c816c9', 'request_id': 'req-4c0c5c84-928f-46b2-aa21-1a69c460146e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:51.914601 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4c0c5c84-928f-46b2-aa21-1a69c460146e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc512d8e2ab848fda424f9eeb6c816c9', 'request_id': 'req-4c0c5c84-928f-46b2-aa21-1a69c460146e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:51.921444 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4c0c5c84-928f-46b2-aa21-1a69c460146e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc512d8e2ab848fda424f9eeb6c816c9', 'request_id': 'req-4c0c5c84-928f-46b2-aa21-1a69c460146e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:51.934210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4c0c5c84-928f-46b2-aa21-1a69c460146e admin admin] The token cache is being invalidate because project dc512d8e2ab848fda424f9eeb6c816c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:29:51.934499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4c0c5c84-928f-46b2-aa21-1a69c460146e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:51.935163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-4c0c5c84-928f-46b2-aa21-1a69c460146e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:51.948651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1172/2332] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:29:51 2022] DELETE /identity/v3/projects/dc512d8e2ab848fda424f9eeb6c816c9 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:29:51.958367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae877bcf-16d7-4770-a15c-66a13affa883 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:52.036428 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35f626-f7ad-4021-b890-88aebbf68f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:52.037139 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35f626-f7ad-4021-b890-88aebbf68f5a admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:52.037438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35f626-f7ad-4021-b890-88aebbf68f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:52.037728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35f626-f7ad-4021-b890-88aebbf68f5a admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:52.038465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35f626-f7ad-4021-b890-88aebbf68f5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:52.040482 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35f626-f7ad-4021-b890-88aebbf68f5a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:52.070394 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1161/2333] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:29:51 2022] POST /identity/v3/users => generated 288 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:29:52.072626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c5c84-928f-46b2-aa21-1a69c460146e admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:52.074158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4c0c5c84-928f-46b2-aa21-1a69c460146e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:52.089073 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0c5c84-928f-46b2-aa21-1a69c460146e admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:52.095177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb21a05-46a9-4285-999c-b4715b7cf1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:52.095177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb21a05-46a9-4285-999c-b4715b7cf1b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:52.095177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb21a05-46a9-4285-999c-b4715b7cf1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:52.095567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb21a05-46a9-4285-999c-b4715b7cf1b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:52.096251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb21a05-46a9-4285-999c-b4715b7cf1b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:52.098676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb21a05-46a9-4285-999c-b4715b7cf1b6 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:52.115074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1173/2334] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:29:52.115946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35f626-f7ad-4021-b890-88aebbf68f5a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:52.130708 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35f626-f7ad-4021-b890-88aebbf68f5a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:52.137111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed94638-2007-47de-aa8d-838fd786c37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:52.138190 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed94638-2007-47de-aa8d-838fd786c37c admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:52.138480 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed94638-2007-47de-aa8d-838fd786c37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:52.140161 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed94638-2007-47de-aa8d-838fd786c37c admin admin] PATH_INFO: `/v3/projects/f85304fbde784932be1b83b21df4873b/users/3c601204ecb140efa2552153a66d6ea1/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:52.160442 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed94638-2007-47de-aa8d-838fd786c37c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85304fbde784932be1b83b21df4873b, user_id=3c601204ecb140efa2552153a66d6ea1, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:52.163684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed94638-2007-47de-aa8d-838fd786c37c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:52.181640 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1162/2335] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:29:52 2022] PUT /identity/v3/projects/f85304fbde784932be1b83b21df4873b/users/3c601204ecb140efa2552153a66d6ea1/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:29:52.185819 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb21a05-46a9-4285-999c-b4715b7cf1b6 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:52.196389 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb21a05-46a9-4285-999c-b4715b7cf1b6 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:52.280422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1dc70a-6475-4d4e-a005-7757073df8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:52.281588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dc70a-6475-4d4e-a005-7757073df8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:52.281588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dc70a-6475-4d4e-a005-7757073df8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:52.281860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dc70a-6475-4d4e-a005-7757073df8f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:52.282577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1dc70a-6475-4d4e-a005-7757073df8f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:52.284274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1dc70a-6475-4d4e-a005-7757073df8f6 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:52.296629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1174/2336] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:29:52.300998 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed94638-2007-47de-aa8d-838fd786c37c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:52.312805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed94638-2007-47de-aa8d-838fd786c37c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:52.318895 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db6621-20e2-463c-aa3b-2f18d1c98f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:52.319818 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db6621-20e2-463c-aa3b-2f18d1c98f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:52.320097 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db6621-20e2-463c-aa3b-2f18d1c98f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:52.320344 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db6621-20e2-463c-aa3b-2f18d1c98f83 admin admin] PATH_INFO: `/v3/projects/f85304fbde784932be1b83b21df4873b/users/3c601204ecb140efa2552153a66d6ea1/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:52.322643 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db6621-20e2-463c-aa3b-2f18d1c98f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85304fbde784932be1b83b21df4873b, user_id=3c601204ecb140efa2552153a66d6ea1, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:52.325431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db6621-20e2-463c-aa3b-2f18d1c98f83 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:52.342129 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1163/2337] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:29:52 2022] PUT /identity/v3/projects/f85304fbde784932be1b83b21df4873b/users/3c601204ecb140efa2552153a66d6ea1/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:52.348207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2c403-20d4-4b19-90eb-f90d07375591 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:52.348568 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2c403-20d4-4b19-90eb-f90d07375591 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:52.348902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2c403-20d4-4b19-90eb-f90d07375591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:52.385811 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-0ba2c403-20d4-4b19-90eb-f90d07375591 None None] MFA Rules not processed for user `3c601204ecb140efa2552153a66d6ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:52.387285 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-0ba2c403-20d4-4b19-90eb-f90d07375591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:52.472503 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1175/2338] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:52 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:52.484349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db6621-20e2-463c-aa3b-2f18d1c98f83 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:52.486706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c5db6621-20e2-463c-aa3b-2f18d1c98f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:52.549372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db6621-20e2-463c-aa3b-2f18d1c98f83 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:52.553254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109df267-5d3a-4f25-8b7f-c984296e33a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:52.554020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109df267-5d3a-4f25-8b7f-c984296e33a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:52.554303 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109df267-5d3a-4f25-8b7f-c984296e33a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:52.554578 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109df267-5d3a-4f25-8b7f-c984296e33a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:52.565397 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109df267-5d3a-4f25-8b7f-c984296e33a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:52.567081 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109df267-5d3a-4f25-8b7f-c984296e33a5 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:52.576998 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1164/2339] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:52 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:52.658699 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2c403-20d4-4b19-90eb-f90d07375591 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:52.670775 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2c403-20d4-4b19-90eb-f90d07375591 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:52.674391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4fc0d-7bfa-4ebd-9926-6be7779809ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:52.675018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4fc0d-7bfa-4ebd-9926-6be7779809ec service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:52.675321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4fc0d-7bfa-4ebd-9926-6be7779809ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:52.675642 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4fc0d-7bfa-4ebd-9926-6be7779809ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:52.677849 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c7a4fc0d-7bfa-4ebd-9926-6be7779809ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:52.697672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:52.741550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4fc0d-7bfa-4ebd-9926-6be7779809ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:52.743195 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4fc0d-7bfa-4ebd-9926-6be7779809ec service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:52.753946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1176/2340] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:52 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:55.552020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109df267-5d3a-4f25-8b7f-c984296e33a5 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:55.578974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:55.617816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109df267-5d3a-4f25-8b7f-c984296e33a5 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:55.622752 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63f61c-c4dc-460e-a509-0e35041e1577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:55.623393 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63f61c-c4dc-460e-a509-0e35041e1577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:55.623667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63f61c-c4dc-460e-a509-0e35041e1577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:55.623937 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63f61c-c4dc-460e-a509-0e35041e1577 admin admin] PATH_INFO: `/v3/users/63f17dc927214bf888d6fc735158a49a` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:55.625113 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63f61c-c4dc-460e-a509-0e35041e1577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63f17dc927214bf888d6fc735158a49a)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:55.627089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63f61c-c4dc-460e-a509-0e35041e1577 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:55.674083 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4b63f61c-c4dc-460e-a509-0e35041e1577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63f17dc927214bf888d6fc735158a49a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:55.674650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4b63f61c-c4dc-460e-a509-0e35041e1577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63f17dc927214bf888d6fc735158a49a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:55.687895 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4b63f61c-c4dc-460e-a509-0e35041e1577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63f17dc927214bf888d6fc735158a49a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:55.696410 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4b63f61c-c4dc-460e-a509-0e35041e1577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63f17dc927214bf888d6fc735158a49a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:55.703621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-4b63f61c-c4dc-460e-a509-0e35041e1577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63f17dc927214bf888d6fc735158a49a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:55.710484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1165/2341] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:29:55 2022] DELETE /identity/v3/users/63f17dc927214bf888d6fc735158a49a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:55.717269 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfae27-2add-4fea-8489-2f581cdf9efb None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:55.717673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfae27-2add-4fea-8489-2f581cdf9efb None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:55.717854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfae27-2add-4fea-8489-2f581cdf9efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:55.738402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-efbfae27-2add-4fea-8489-2f581cdf9efb None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:55.739624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-efbfae27-2add-4fea-8489-2f581cdf9efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:55.803074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1177/2342] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:55.816149 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63f61c-c4dc-460e-a509-0e35041e1577 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:55.817938 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4b63f61c-c4dc-460e-a509-0e35041e1577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:55.888788 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63f61c-c4dc-460e-a509-0e35041e1577 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:55.893672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9cc0a-b500-48a5-a930-1805642840c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:55.894473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9cc0a-b500-48a5-a930-1805642840c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:55.894750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9cc0a-b500-48a5-a930-1805642840c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:55.895017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9cc0a-b500-48a5-a930-1805642840c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:55.905994 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9cc0a-b500-48a5-a930-1805642840c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:55.907459 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9cc0a-b500-48a5-a930-1805642840c4 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:55.918032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1166/2343] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:56.289804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfae27-2add-4fea-8489-2f581cdf9efb None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:56.289804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-efbfae27-2add-4fea-8489-2f581cdf9efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:56.303434 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfae27-2add-4fea-8489-2f581cdf9efb None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:56.309617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236063f3-b3e6-4102-b587-a18b71370cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:56.310724 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236063f3-b3e6-4102-b587-a18b71370cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:56.310998 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236063f3-b3e6-4102-b587-a18b71370cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:56.311284 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236063f3-b3e6-4102-b587-a18b71370cb9 admin admin] PATH_INFO: `/v3/projects/7dea182791a346868d0d7382171a3d4d` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:56.312551 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236063f3-b3e6-4102-b587-a18b71370cb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dea182791a346868d0d7382171a3d4d)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:56.314575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236063f3-b3e6-4102-b587-a18b71370cb9 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:56.367084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-236063f3-b3e6-4102-b587-a18b71370cb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dea182791a346868d0d7382171a3d4d', 'request_id': 'req-236063f3-b3e6-4102-b587-a18b71370cb9'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:56.375778 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-236063f3-b3e6-4102-b587-a18b71370cb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dea182791a346868d0d7382171a3d4d', 'request_id': 'req-236063f3-b3e6-4102-b587-a18b71370cb9'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:56.383200 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-236063f3-b3e6-4102-b587-a18b71370cb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dea182791a346868d0d7382171a3d4d', 'request_id': 'req-236063f3-b3e6-4102-b587-a18b71370cb9'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:56.394733 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-236063f3-b3e6-4102-b587-a18b71370cb9 admin admin] The token cache is being invalidate because project 7dea182791a346868d0d7382171a3d4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:29:56.395023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-236063f3-b3e6-4102-b587-a18b71370cb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:56.395686 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-236063f3-b3e6-4102-b587-a18b71370cb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:56.397287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1178/2344] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:29:56 2022] DELETE /identity/v3/projects/7dea182791a346868d0d7382171a3d4d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:56.401369 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9cc0a-b500-48a5-a930-1805642840c4 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:56.411888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9cc0a-b500-48a5-a930-1805642840c4 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:56.454765 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:56.471423 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bcaeb-2645-4ed8-8187-d65c4ac97407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:56.472006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bcaeb-2645-4ed8-8187-d65c4ac97407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:56.472337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bcaeb-2645-4ed8-8187-d65c4ac97407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:56.472611 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bcaeb-2645-4ed8-8187-d65c4ac97407 admin admin] PATH_INFO: `/v3/users/3c601204ecb140efa2552153a66d6ea1` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:56.473794 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bcaeb-2645-4ed8-8187-d65c4ac97407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c601204ecb140efa2552153a66d6ea1)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:56.476065 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bcaeb-2645-4ed8-8187-d65c4ac97407 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:56.531301 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-450bcaeb-2645-4ed8-8187-d65c4ac97407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c601204ecb140efa2552153a66d6ea1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:56.531951 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-450bcaeb-2645-4ed8-8187-d65c4ac97407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c601204ecb140efa2552153a66d6ea1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:56.544649 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-450bcaeb-2645-4ed8-8187-d65c4ac97407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c601204ecb140efa2552153a66d6ea1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:56.554090 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-450bcaeb-2645-4ed8-8187-d65c4ac97407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c601204ecb140efa2552153a66d6ea1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:56.561568 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-450bcaeb-2645-4ed8-8187-d65c4ac97407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c601204ecb140efa2552153a66d6ea1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:56.568199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1167/2345] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:29:56 2022] DELETE /identity/v3/users/3c601204ecb140efa2552153a66d6ea1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:56.849935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236063f3-b3e6-4102-b587-a18b71370cb9 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:56.851913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-236063f3-b3e6-4102-b587-a18b71370cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:56.869509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236063f3-b3e6-4102-b587-a18b71370cb9 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:56.918362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:56.935812 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a790c580-929b-44de-b72d-2b01f33413c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:56.936505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790c580-929b-44de-b72d-2b01f33413c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:56.936789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790c580-929b-44de-b72d-2b01f33413c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:56.937068 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a790c580-929b-44de-b72d-2b01f33413c2 admin admin] PATH_INFO: `/v3/projects/f85304fbde784932be1b83b21df4873b` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:56.938310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790c580-929b-44de-b72d-2b01f33413c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f85304fbde784932be1b83b21df4873b)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:56.940279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a790c580-929b-44de-b72d-2b01f33413c2 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:56.988991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a790c580-929b-44de-b72d-2b01f33413c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f85304fbde784932be1b83b21df4873b', 'request_id': 'req-a790c580-929b-44de-b72d-2b01f33413c2'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:56.997083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a790c580-929b-44de-b72d-2b01f33413c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f85304fbde784932be1b83b21df4873b', 'request_id': 'req-a790c580-929b-44de-b72d-2b01f33413c2'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:57.004080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a790c580-929b-44de-b72d-2b01f33413c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f85304fbde784932be1b83b21df4873b', 'request_id': 'req-a790c580-929b-44de-b72d-2b01f33413c2'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:57.014698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a790c580-929b-44de-b72d-2b01f33413c2 admin admin] The token cache is being invalidate because project f85304fbde784932be1b83b21df4873b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:29:57.014986 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a790c580-929b-44de-b72d-2b01f33413c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:57.015624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a790c580-929b-44de-b72d-2b01f33413c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:57.017156 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1179/2346] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:29:56 2022] DELETE /identity/v3/projects/f85304fbde784932be1b83b21df4873b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:57.022596 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1963154-3dd4-4aee-bb84-de4eef758309 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:57.022884 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1963154-3dd4-4aee-bb84-de4eef758309 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:57.023157 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1963154-3dd4-4aee-bb84-de4eef758309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:57.048246 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-a1963154-3dd4-4aee-bb84-de4eef758309 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:57.049476 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a1963154-3dd4-4aee-bb84-de4eef758309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:57.114736 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1168/2347] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:57.123014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad419ed-e760-4148-b05d-626b094e5ed1 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:57.123379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad419ed-e760-4148-b05d-626b094e5ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:57.123800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad419ed-e760-4148-b05d-626b094e5ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:57.145854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-9ad419ed-e760-4148-b05d-626b094e5ed1 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:57.147351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9ad419ed-e760-4148-b05d-626b094e5ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:57.156484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1180/2348] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:57.161086 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1963154-3dd4-4aee-bb84-de4eef758309 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:57.171098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1963154-3dd4-4aee-bb84-de4eef758309 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:57.175967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c50c0-235f-4fcd-803f-e5b871c3bd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:57.176593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c50c0-235f-4fcd-803f-e5b871c3bd9d admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:57.176862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c50c0-235f-4fcd-803f-e5b871c3bd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:57.177139 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c50c0-235f-4fcd-803f-e5b871c3bd9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:57.177896 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c50c0-235f-4fcd-803f-e5b871c3bd9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:29:57.178187 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c50c0-235f-4fcd-803f-e5b871c3bd9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:57.179809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c50c0-235f-4fcd-803f-e5b871c3bd9d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:57.191493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1169/2349] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:29:57.197643 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad419ed-e760-4148-b05d-626b094e5ed1 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:57.212950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad419ed-e760-4148-b05d-626b094e5ed1 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:57.220707 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f48eb7-26f3-4bff-8e79-9c15e24be293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:57.221660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f48eb7-26f3-4bff-8e79-9c15e24be293 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:57.222086 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f48eb7-26f3-4bff-8e79-9c15e24be293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:57.222455 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f48eb7-26f3-4bff-8e79-9c15e24be293 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:57.223124 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f48eb7-26f3-4bff-8e79-9c15e24be293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:57.225065 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f48eb7-26f3-4bff-8e79-9c15e24be293 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:57.257668 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1181/2350] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:29:57 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:29:57.261773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c50c0-235f-4fcd-803f-e5b871c3bd9d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:57.272302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c50c0-235f-4fcd-803f-e5b871c3bd9d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:57.323047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:57.337025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7012a7-5dbf-4b96-8475-22cb1a9451a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:57.337660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7012a7-5dbf-4b96-8475-22cb1a9451a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:57.337999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7012a7-5dbf-4b96-8475-22cb1a9451a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:57.338275 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7012a7-5dbf-4b96-8475-22cb1a9451a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:57.338959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7012a7-5dbf-4b96-8475-22cb1a9451a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:57.340924 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7012a7-5dbf-4b96-8475-22cb1a9451a4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:57.364474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1170/2351] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:29:57 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:29:57.369996 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f48eb7-26f3-4bff-8e79-9c15e24be293 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:57.383121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f48eb7-26f3-4bff-8e79-9c15e24be293 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:57.388618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19856e58-a689-4a28-ae40-1c3b38dc4973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:57.389249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19856e58-a689-4a28-ae40-1c3b38dc4973 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:57.389521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19856e58-a689-4a28-ae40-1c3b38dc4973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:57.389824 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19856e58-a689-4a28-ae40-1c3b38dc4973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:57.390537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19856e58-a689-4a28-ae40-1c3b38dc4973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:57.392130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19856e58-a689-4a28-ae40-1c3b38dc4973 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:57.403910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1182/2352] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:29:57.408133 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7012a7-5dbf-4b96-8475-22cb1a9451a4 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:57.422176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7012a7-5dbf-4b96-8475-22cb1a9451a4 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:57.427977 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61e8de-64ce-4e91-ba04-cc0299a371c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:57.428614 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61e8de-64ce-4e91-ba04-cc0299a371c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:57.428943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61e8de-64ce-4e91-ba04-cc0299a371c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:57.429162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61e8de-64ce-4e91-ba04-cc0299a371c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:57.429909 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f61e8de-64ce-4e91-ba04-cc0299a371c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:57.431532 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f61e8de-64ce-4e91-ba04-cc0299a371c7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:57.449308 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1171/2353] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:29:57.453962 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19856e58-a689-4a28-ae40-1c3b38dc4973 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:57.464688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19856e58-a689-4a28-ae40-1c3b38dc4973 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:57.469687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09f172-2138-48ab-95c0-4c3208e2a608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:57.470381 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09f172-2138-48ab-95c0-4c3208e2a608 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:57.470656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09f172-2138-48ab-95c0-4c3208e2a608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:57.470944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09f172-2138-48ab-95c0-4c3208e2a608 admin admin] PATH_INFO: `/v3/projects/e1b67487716148808b321ce1a5ffcd85/users/9ca921b5aa0b4d94ae743db184cd5b34/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:57.487224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09f172-2138-48ab-95c0-4c3208e2a608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1b67487716148808b321ce1a5ffcd85, user_id=9ca921b5aa0b4d94ae743db184cd5b34, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:57.490501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09f172-2138-48ab-95c0-4c3208e2a608 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:57.507770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1183/2354] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:29:57 2022] PUT /identity/v3/projects/e1b67487716148808b321ce1a5ffcd85/users/9ca921b5aa0b4d94ae743db184cd5b34/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:57.523397 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdc8e8-a71d-4f66-996f-5e5260f07ee9 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:57.523685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdc8e8-a71d-4f66-996f-5e5260f07ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:57.523959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdc8e8-a71d-4f66-996f-5e5260f07ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:57.553174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2dbdc8e8-a71d-4f66-996f-5e5260f07ee9 None None] MFA Rules not processed for user `9ca921b5aa0b4d94ae743db184cd5b34`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:57.554435 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2dbdc8e8-a71d-4f66-996f-5e5260f07ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:57.625124 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1172/2355] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:57.629809 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09f172-2138-48ab-95c0-4c3208e2a608 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:57.692920 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09f172-2138-48ab-95c0-4c3208e2a608 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:57.699215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c06b9f-fdf8-4cfc-ab8d-694cc8b40043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:57.699863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c06b9f-fdf8-4cfc-ab8d-694cc8b40043 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:57.700139 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c06b9f-fdf8-4cfc-ab8d-694cc8b40043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:57.700412 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c06b9f-fdf8-4cfc-ab8d-694cc8b40043 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:57.701049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c06b9f-fdf8-4cfc-ab8d-694cc8b40043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:57.703033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c06b9f-fdf8-4cfc-ab8d-694cc8b40043 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:57.720208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:57.738027 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1184/2356] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:29:57 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:29:57.742130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdc8e8-a71d-4f66-996f-5e5260f07ee9 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:57.753997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdc8e8-a71d-4f66-996f-5e5260f07ee9 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:57.822485 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d555b-c23c-4f15-a1c6-935e760cc6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:57.823108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d555b-c23c-4f15-a1c6-935e760cc6a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:57.823420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d555b-c23c-4f15-a1c6-935e760cc6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:57.823696 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d555b-c23c-4f15-a1c6-935e760cc6a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:57.824349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d555b-c23c-4f15-a1c6-935e760cc6a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:57.826372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d555b-c23c-4f15-a1c6-935e760cc6a5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:57.848147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1173/2357] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:29:57 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:29:57.852320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c06b9f-fdf8-4cfc-ab8d-694cc8b40043 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:57.862764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c06b9f-fdf8-4cfc-ab8d-694cc8b40043 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:57.868038 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3afa1b-b666-4a4d-9f74-30446e9f3882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:57.868674 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3afa1b-b666-4a4d-9f74-30446e9f3882 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:57.868947 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3afa1b-b666-4a4d-9f74-30446e9f3882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:57.869215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3afa1b-b666-4a4d-9f74-30446e9f3882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:57.869949 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3afa1b-b666-4a4d-9f74-30446e9f3882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:57.871565 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3afa1b-b666-4a4d-9f74-30446e9f3882 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:57.882840 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1185/2358] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:29:57.887169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d555b-c23c-4f15-a1c6-935e760cc6a5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:57.898213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d555b-c23c-4f15-a1c6-935e760cc6a5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:57.904602 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6210ad-f06a-4f89-b7e1-dd699e8e0da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:57.905358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6210ad-f06a-4f89-b7e1-dd699e8e0da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:57.905604 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6210ad-f06a-4f89-b7e1-dd699e8e0da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:57.906048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6210ad-f06a-4f89-b7e1-dd699e8e0da9 admin admin] PATH_INFO: `/v3/projects/97313969825741399138c132015afb1a/users/0c2717cddc294826b9179899018f9280/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:57.923966 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6210ad-f06a-4f89-b7e1-dd699e8e0da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97313969825741399138c132015afb1a, user_id=0c2717cddc294826b9179899018f9280, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:57.926993 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6210ad-f06a-4f89-b7e1-dd699e8e0da9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:57.942300 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1174/2359] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:29:57 2022] PUT /identity/v3/projects/97313969825741399138c132015afb1a/users/0c2717cddc294826b9179899018f9280/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:57.946359 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3afa1b-b666-4a4d-9f74-30446e9f3882 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:57.957180 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3afa1b-b666-4a4d-9f74-30446e9f3882 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:58.017323 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7a58d-0ad7-4f13-aeb0-967eaacf321d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:58.018114 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7a58d-0ad7-4f13-aeb0-967eaacf321d admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:58.018403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7a58d-0ad7-4f13-aeb0-967eaacf321d admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:58.018677 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7a58d-0ad7-4f13-aeb0-967eaacf321d admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:58.019418 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7a58d-0ad7-4f13-aeb0-967eaacf321d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:58.021208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7a58d-0ad7-4f13-aeb0-967eaacf321d admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:58.033222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1186/2360] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:29:58.037350 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6210ad-f06a-4f89-b7e1-dd699e8e0da9 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:58.048198 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6210ad-f06a-4f89-b7e1-dd699e8e0da9 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:58.053709 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614113c-6b91-4089-8131-580808f38219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:58.054428 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614113c-6b91-4089-8131-580808f38219 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:58.054702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614113c-6b91-4089-8131-580808f38219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:58.054969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b614113c-6b91-4089-8131-580808f38219 admin admin] PATH_INFO: `/v3/projects/97313969825741399138c132015afb1a/users/0c2717cddc294826b9179899018f9280/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:58.057306 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614113c-6b91-4089-8131-580808f38219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97313969825741399138c132015afb1a, user_id=0c2717cddc294826b9179899018f9280, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:58.060063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b614113c-6b91-4089-8131-580808f38219 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:58.071472 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1175/2361] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:29:58 2022] PUT /identity/v3/projects/97313969825741399138c132015afb1a/users/0c2717cddc294826b9179899018f9280/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:58.082513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fcd08-ead4-4a67-915d-419cfa28bc19 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:58.082900 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fcd08-ead4-4a67-915d-419cfa28bc19 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:58.083145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182fcd08-ead4-4a67-915d-419cfa28bc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:58.115321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-182fcd08-ead4-4a67-915d-419cfa28bc19 None None] MFA Rules not processed for user `0c2717cddc294826b9179899018f9280`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:58.116555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-182fcd08-ead4-4a67-915d-419cfa28bc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:58.191717 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1187/2362] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:58.204231 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614113c-6b91-4089-8131-580808f38219 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:58.206310 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b614113c-6b91-4089-8131-580808f38219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:58.221568 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:58.275520 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b614113c-6b91-4089-8131-580808f38219 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:58.279895 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb37e0a-2e42-4329-a9c9-f4988b7035d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:58.280601 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb37e0a-2e42-4329-a9c9-f4988b7035d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:58.280944 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb37e0a-2e42-4329-a9c9-f4988b7035d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:58.281296 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb37e0a-2e42-4329-a9c9-f4988b7035d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:58.294414 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb37e0a-2e42-4329-a9c9-f4988b7035d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:58.295989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb37e0a-2e42-4329-a9c9-f4988b7035d0 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:58.307646 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1176/2363] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:58 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:58.661138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fcd08-ead4-4a67-915d-419cfa28bc19 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:58.670357 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182fcd08-ead4-4a67-915d-419cfa28bc19 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:58.675874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3a9cd-da03-4ee7-bcb4-d89a083ac2cc tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c2717cddc294826b9179899018f9280', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97313969825741399138c132015afb1a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:58.676725 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3a9cd-da03-4ee7-bcb4-d89a083ac2cc tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:58.677012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3a9cd-da03-4ee7-bcb4-d89a083ac2cc tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:58.677290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3a9cd-da03-4ee7-bcb4-d89a083ac2cc tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:58.678060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3a9cd-da03-4ee7-bcb4-d89a083ac2cc tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:29:58.678376 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3a9cd-da03-4ee7-bcb4-d89a083ac2cc tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:58.680039 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3a9cd-da03-4ee7-bcb4-d89a083ac2cc tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:58.692139 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1188/2364] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:29:58.696578 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb37e0a-2e42-4329-a9c9-f4988b7035d0 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:58.706404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb37e0a-2e42-4329-a9c9-f4988b7035d0 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:58.712111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94107569-2c4f-4ab2-87a2-f9372e4e8c22 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c2717cddc294826b9179899018f9280', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97313969825741399138c132015afb1a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:58.712747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94107569-2c4f-4ab2-87a2-f9372e4e8c22 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:58.713026 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94107569-2c4f-4ab2-87a2-f9372e4e8c22 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:58.713297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94107569-2c4f-4ab2-87a2-f9372e4e8c22 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:58.714011 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94107569-2c4f-4ab2-87a2-f9372e4e8c22 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:58.715991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94107569-2c4f-4ab2-87a2-f9372e4e8c22 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:58.747799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1177/2365] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:29:58 2022] POST /identity/v3/projects => generated 355 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:29:58.751929 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3a9cd-da03-4ee7-bcb4-d89a083ac2cc tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:58.762274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3a9cd-da03-4ee7-bcb4-d89a083ac2cc tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:58.813391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:58.823654 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483aca1-163d-4de4-b622-2e3b2ebb64aa tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c2717cddc294826b9179899018f9280', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97313969825741399138c132015afb1a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:58.824295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483aca1-163d-4de4-b622-2e3b2ebb64aa tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:58.825185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483aca1-163d-4de4-b622-2e3b2ebb64aa tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:58.825185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483aca1-163d-4de4-b622-2e3b2ebb64aa tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:58.825540 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483aca1-163d-4de4-b622-2e3b2ebb64aa tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:29:58.825861 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483aca1-163d-4de4-b622-2e3b2ebb64aa tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:58.827479 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483aca1-163d-4de4-b622-2e3b2ebb64aa tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:58.841121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1189/2366] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:29:58.897662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94107569-2c4f-4ab2-87a2-f9372e4e8c22 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:58.911644 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94107569-2c4f-4ab2-87a2-f9372e4e8c22 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:58.917413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6043c-5428-47a6-b49f-2c0f3ffe4126 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c2717cddc294826b9179899018f9280', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97313969825741399138c132015afb1a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:58.918094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6043c-5428-47a6-b49f-2c0f3ffe4126 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:58.918375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6043c-5428-47a6-b49f-2c0f3ffe4126 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:58.918640 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6043c-5428-47a6-b49f-2c0f3ffe4126 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] PATH_INFO: `/v3/projects/201202b202ac4fffbc12eede76453643` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:58.919972 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6043c-5428-47a6-b49f-2c0f3ffe4126 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] RBAC: Authorizing `identity:delete_project(project_id=201202b202ac4fffbc12eede76453643)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:58.921943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6043c-5428-47a6-b49f-2c0f3ffe4126 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:58.976982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5bc6043c-5428-47a6-b49f-2c0f3ffe4126 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201202b202ac4fffbc12eede76453643', 'request_id': 'req-5bc6043c-5428-47a6-b49f-2c0f3ffe4126'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:58.982780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5bc6043c-5428-47a6-b49f-2c0f3ffe4126 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201202b202ac4fffbc12eede76453643', 'request_id': 'req-5bc6043c-5428-47a6-b49f-2c0f3ffe4126'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:58.992235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5bc6043c-5428-47a6-b49f-2c0f3ffe4126 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201202b202ac4fffbc12eede76453643', 'request_id': 'req-5bc6043c-5428-47a6-b49f-2c0f3ffe4126'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:59.005534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5bc6043c-5428-47a6-b49f-2c0f3ffe4126 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] The token cache is being invalidate because project 201202b202ac4fffbc12eede76453643 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:29:59.005534 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5bc6043c-5428-47a6-b49f-2c0f3ffe4126 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:59.006213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5bc6043c-5428-47a6-b49f-2c0f3ffe4126 tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:59.007778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1178/2367] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:29:58 2022] DELETE /identity/v3/projects/201202b202ac4fffbc12eede76453643 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:59.235566 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483aca1-163d-4de4-b622-2e3b2ebb64aa tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:59.238662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c483aca1-163d-4de4-b622-2e3b2ebb64aa tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:59.309410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483aca1-163d-4de4-b622-2e3b2ebb64aa tempest-VolumeQuotaClassesTest-874985966 tempest-VolumeQuotaClassesTest-874985966-project-admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:59.372057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81387fa7-ae70-466d-97b7-f91acddedc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:59.372720 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81387fa7-ae70-466d-97b7-f91acddedc8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:59.372999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81387fa7-ae70-466d-97b7-f91acddedc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:59.373293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81387fa7-ae70-466d-97b7-f91acddedc8e admin admin] PATH_INFO: `/v3/users/9ca921b5aa0b4d94ae743db184cd5b34` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:59.374524 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81387fa7-ae70-466d-97b7-f91acddedc8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca921b5aa0b4d94ae743db184cd5b34)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:59.376550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81387fa7-ae70-466d-97b7-f91acddedc8e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:59.430168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-81387fa7-ae70-466d-97b7-f91acddedc8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca921b5aa0b4d94ae743db184cd5b34'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:59.441301 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-81387fa7-ae70-466d-97b7-f91acddedc8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca921b5aa0b4d94ae743db184cd5b34'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:59.442110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-81387fa7-ae70-466d-97b7-f91acddedc8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca921b5aa0b4d94ae743db184cd5b34'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:59.449217 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-81387fa7-ae70-466d-97b7-f91acddedc8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca921b5aa0b4d94ae743db184cd5b34'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:59.461435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-81387fa7-ae70-466d-97b7-f91acddedc8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca921b5aa0b4d94ae743db184cd5b34'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:29:59.467382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1190/2368] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:29:59 2022] DELETE /identity/v3/users/9ca921b5aa0b4d94ae743db184cd5b34 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:29:59.471983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049117d-14d7-4570-afa5-088b7e6459b7 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:59.472365 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049117d-14d7-4570-afa5-088b7e6459b7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:59.472722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049117d-14d7-4570-afa5-088b7e6459b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:59.492229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-0049117d-14d7-4570-afa5-088b7e6459b7 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:59.493441 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0049117d-14d7-4570-afa5-088b7e6459b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:59.554503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1179/2369] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:29:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:59.566734 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81387fa7-ae70-466d-97b7-f91acddedc8e admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:59.567887 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-81387fa7-ae70-466d-97b7-f91acddedc8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:59.584443 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:59.633094 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81387fa7-ae70-466d-97b7-f91acddedc8e admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:59.637375 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e434e0-4533-4692-8754-1f7ea0b03572 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:59.638039 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e434e0-4533-4692-8754-1f7ea0b03572 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:59.638317 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e434e0-4533-4692-8754-1f7ea0b03572 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:59.638581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e434e0-4533-4692-8754-1f7ea0b03572 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:59.648999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e434e0-4533-4692-8754-1f7ea0b03572 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:59.650490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e434e0-4533-4692-8754-1f7ea0b03572 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:59.660210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1191/2370] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:29:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:59.954802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049117d-14d7-4570-afa5-088b7e6459b7 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:59.956835 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0049117d-14d7-4570-afa5-088b7e6459b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:59.980024 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049117d-14d7-4570-afa5-088b7e6459b7 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:59.986740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e8cb2-f44d-4d9c-812d-f83a13b08870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:59.987425 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8cb2-f44d-4d9c-812d-f83a13b08870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:59.987795 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8cb2-f44d-4d9c-812d-f83a13b08870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:59.988027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e8cb2-f44d-4d9c-812d-f83a13b08870 admin admin] PATH_INFO: `/v3/projects/e1b67487716148808b321ce1a5ffcd85` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:59.989539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e8cb2-f44d-4d9c-812d-f83a13b08870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1b67487716148808b321ce1a5ffcd85)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:59.991562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e8cb2-f44d-4d9c-812d-f83a13b08870 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:00.004667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:00.044810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-da6e8cb2-f44d-4d9c-812d-f83a13b08870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1b67487716148808b321ce1a5ffcd85', 'request_id': 'req-da6e8cb2-f44d-4d9c-812d-f83a13b08870'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:00.050382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-da6e8cb2-f44d-4d9c-812d-f83a13b08870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1b67487716148808b321ce1a5ffcd85', 'request_id': 'req-da6e8cb2-f44d-4d9c-812d-f83a13b08870'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:00.059939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-da6e8cb2-f44d-4d9c-812d-f83a13b08870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1b67487716148808b321ce1a5ffcd85', 'request_id': 'req-da6e8cb2-f44d-4d9c-812d-f83a13b08870'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:00.072385 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-da6e8cb2-f44d-4d9c-812d-f83a13b08870 admin admin] The token cache is being invalidate because project e1b67487716148808b321ce1a5ffcd85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:30:00.072668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-da6e8cb2-f44d-4d9c-812d-f83a13b08870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:00.073291 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-da6e8cb2-f44d-4d9c-812d-f83a13b08870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:00.074863 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1180/2371] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:29:59 2022] DELETE /identity/v3/projects/e1b67487716148808b321ce1a5ffcd85 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:00.080008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e434e0-4533-4692-8754-1f7ea0b03572 service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:00.094183 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e434e0-4533-4692-8754-1f7ea0b03572 service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:00.154739 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f6744-2aa5-4151-a9f4-cc4f4a601db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:00.155427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f6744-2aa5-4151-a9f4-cc4f4a601db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:00.155742 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f6744-2aa5-4151-a9f4-cc4f4a601db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:00.156098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f6744-2aa5-4151-a9f4-cc4f4a601db5 admin admin] PATH_INFO: `/v3/users/0c2717cddc294826b9179899018f9280` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:00.157416 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f6744-2aa5-4151-a9f4-cc4f4a601db5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c2717cddc294826b9179899018f9280)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:00.159456 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f6744-2aa5-4151-a9f4-cc4f4a601db5 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:00.223479 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-f37f6744-2aa5-4151-a9f4-cc4f4a601db5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c2717cddc294826b9179899018f9280'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:00.233033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-f37f6744-2aa5-4151-a9f4-cc4f4a601db5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c2717cddc294826b9179899018f9280'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:00.233648 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-f37f6744-2aa5-4151-a9f4-cc4f4a601db5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c2717cddc294826b9179899018f9280'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:00.242279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-f37f6744-2aa5-4151-a9f4-cc4f4a601db5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c2717cddc294826b9179899018f9280'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:00.258734 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-f37f6744-2aa5-4151-a9f4-cc4f4a601db5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c2717cddc294826b9179899018f9280'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:00.265332 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1192/2372] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:30:00 2022] DELETE /identity/v3/users/0c2717cddc294826b9179899018f9280 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:00.592759 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e8cb2-f44d-4d9c-812d-f83a13b08870 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:00.594397 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-da6e8cb2-f44d-4d9c-812d-f83a13b08870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:00.608820 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e8cb2-f44d-4d9c-812d-f83a13b08870 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:00.674873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218397b-fbe3-4a40-9c21-9b90b57a09ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:00.675599 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218397b-fbe3-4a40-9c21-9b90b57a09ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:00.675922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218397b-fbe3-4a40-9c21-9b90b57a09ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:00.676179 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218397b-fbe3-4a40-9c21-9b90b57a09ab admin admin] PATH_INFO: `/v3/projects/97313969825741399138c132015afb1a` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:00.677411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218397b-fbe3-4a40-9c21-9b90b57a09ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=97313969825741399138c132015afb1a)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:00.679426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218397b-fbe3-4a40-9c21-9b90b57a09ab admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:00.736295 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2218397b-fbe3-4a40-9c21-9b90b57a09ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97313969825741399138c132015afb1a', 'request_id': 'req-2218397b-fbe3-4a40-9c21-9b90b57a09ab'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:00.742638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2218397b-fbe3-4a40-9c21-9b90b57a09ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97313969825741399138c132015afb1a', 'request_id': 'req-2218397b-fbe3-4a40-9c21-9b90b57a09ab'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:00.763432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2218397b-fbe3-4a40-9c21-9b90b57a09ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97313969825741399138c132015afb1a', 'request_id': 'req-2218397b-fbe3-4a40-9c21-9b90b57a09ab'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:00.777629 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2218397b-fbe3-4a40-9c21-9b90b57a09ab admin admin] The token cache is being invalidate because project 97313969825741399138c132015afb1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:30:00.777956 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2218397b-fbe3-4a40-9c21-9b90b57a09ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:00.778743 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-2218397b-fbe3-4a40-9c21-9b90b57a09ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:00.780791 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1181/2373] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:30:00 2022] DELETE /identity/v3/projects/97313969825741399138c132015afb1a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:00.787890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d7f38-81d5-4473-acde-92a509338e48 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:00.788227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d7f38-81d5-4473-acde-92a509338e48 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:00.788595 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d7f38-81d5-4473-acde-92a509338e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:00.818242 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-bf9d7f38-81d5-4473-acde-92a509338e48 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:00.819491 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-bf9d7f38-81d5-4473-acde-92a509338e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:00.851651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:00.890332 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1193/2374] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:00.899771 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1b1e2-c6df-4b1d-8b5f-88ac0560f76c None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:00.900134 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1b1e2-c6df-4b1d-8b5f-88ac0560f76c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:00.900295 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1b1e2-c6df-4b1d-8b5f-88ac0560f76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:00.920582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-eed1b1e2-c6df-4b1d-8b5f-88ac0560f76c None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:00.921935 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-eed1b1e2-c6df-4b1d-8b5f-88ac0560f76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:00.930378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1182/2375] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:00.934870 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d7f38-81d5-4473-acde-92a509338e48 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:00.945769 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d7f38-81d5-4473-acde-92a509338e48 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:00.950554 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906dff4d-bac4-4126-8710-2ea6e6dfab14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:00.951183 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dff4d-bac4-4126-8710-2ea6e6dfab14 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:00.951464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dff4d-bac4-4126-8710-2ea6e6dfab14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:00.951738 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dff4d-bac4-4126-8710-2ea6e6dfab14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:00.952448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906dff4d-bac4-4126-8710-2ea6e6dfab14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:30:00.952750 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906dff4d-bac4-4126-8710-2ea6e6dfab14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:00.954424 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906dff4d-bac4-4126-8710-2ea6e6dfab14 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:00.970447 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1194/2376] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:30:00.971688 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1b1e2-c6df-4b1d-8b5f-88ac0560f76c None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:00.981553 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1b1e2-c6df-4b1d-8b5f-88ac0560f76c None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:00.986443 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857d96e-f547-4117-836a-5c3e539243cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:00.987100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857d96e-f547-4117-836a-5c3e539243cf admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:00.987380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857d96e-f547-4117-836a-5c3e539243cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:00.987657 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857d96e-f547-4117-836a-5c3e539243cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:00.988321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857d96e-f547-4117-836a-5c3e539243cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:00.990324 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857d96e-f547-4117-836a-5c3e539243cf admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:01.013401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:01.022045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1183/2377] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:30:00 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:30:01.026282 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906dff4d-bac4-4126-8710-2ea6e6dfab14 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:01.037759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906dff4d-bac4-4126-8710-2ea6e6dfab14 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:01.097550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071ece4-1834-417c-9206-2b068386d1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:01.098201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071ece4-1834-417c-9206-2b068386d1fe admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:01.098481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071ece4-1834-417c-9206-2b068386d1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:01.098750 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071ece4-1834-417c-9206-2b068386d1fe admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:01.099450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071ece4-1834-417c-9206-2b068386d1fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:01.101388 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071ece4-1834-417c-9206-2b068386d1fe admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:01.125003 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1195/2378] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:30:01 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:30:01.129185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857d96e-f547-4117-836a-5c3e539243cf admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:01.144131 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857d96e-f547-4117-836a-5c3e539243cf admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:01.149370 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099115b-02ca-42c0-9834-979bad576873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:01.150203 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099115b-02ca-42c0-9834-979bad576873 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:01.150483 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099115b-02ca-42c0-9834-979bad576873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:01.150752 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099115b-02ca-42c0-9834-979bad576873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:01.151465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099115b-02ca-42c0-9834-979bad576873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:01.153105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8099115b-02ca-42c0-9834-979bad576873 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:01.164381 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1184/2379] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:30:01.168488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071ece4-1834-417c-9206-2b068386d1fe admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:01.178834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071ece4-1834-417c-9206-2b068386d1fe admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:01.190759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e3342-034e-4fad-9c29-5c83569b8919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:01.191813 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e3342-034e-4fad-9c29-5c83569b8919 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:01.192116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e3342-034e-4fad-9c29-5c83569b8919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:01.192384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e3342-034e-4fad-9c29-5c83569b8919 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:01.193119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e3342-034e-4fad-9c29-5c83569b8919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:01.194802 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e3342-034e-4fad-9c29-5c83569b8919 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:01.212241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1196/2380] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:30:01.214319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099115b-02ca-42c0-9834-979bad576873 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:01.233402 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8099115b-02ca-42c0-9834-979bad576873 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:01.239691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b126ff-5420-494e-81ac-b76107611ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:01.240434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b126ff-5420-494e-81ac-b76107611ac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:01.240897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b126ff-5420-494e-81ac-b76107611ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:01.241180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b126ff-5420-494e-81ac-b76107611ac6 admin admin] PATH_INFO: `/v3/projects/a10dd91271c34beaa1d20d736759c372/users/0c7c239805ee43a698de91e0739f33cd/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:01.259038 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b126ff-5420-494e-81ac-b76107611ac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a10dd91271c34beaa1d20d736759c372, user_id=0c7c239805ee43a698de91e0739f33cd, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:01.263840 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b126ff-5420-494e-81ac-b76107611ac6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:01.282175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1185/2381] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:30:01 2022] PUT /identity/v3/projects/a10dd91271c34beaa1d20d736759c372/users/0c7c239805ee43a698de91e0739f33cd/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:01.293126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963e0e4-2482-4df1-86fb-1e298f0ac058 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:01.293732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963e0e4-2482-4df1-86fb-1e298f0ac058 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:01.293862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963e0e4-2482-4df1-86fb-1e298f0ac058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:01.330127 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-4963e0e4-2482-4df1-86fb-1e298f0ac058 None None] MFA Rules not processed for user `0c7c239805ee43a698de91e0739f33cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:01.332315 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4963e0e4-2482-4df1-86fb-1e298f0ac058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:01.431837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1197/2382] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:30:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:01.437333 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b126ff-5420-494e-81ac-b76107611ac6 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:01.517121 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b126ff-5420-494e-81ac-b76107611ac6 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:01.522311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0face3c4-f1a1-4638-8303-789b649fee6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:01.523052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0face3c4-f1a1-4638-8303-789b649fee6e admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:01.523431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0face3c4-f1a1-4638-8303-789b649fee6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:01.523598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0face3c4-f1a1-4638-8303-789b649fee6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:01.524231 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0face3c4-f1a1-4638-8303-789b649fee6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:01.526204 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0face3c4-f1a1-4638-8303-789b649fee6e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:01.564166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1186/2383] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:30:01 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:30:01.565422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963e0e4-2482-4df1-86fb-1e298f0ac058 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:01.578747 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963e0e4-2482-4df1-86fb-1e298f0ac058 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:01.624930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:01.658871 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08b77a-3927-4f08-92f2-e4f3f2739d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:01.659740 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08b77a-3927-4f08-92f2-e4f3f2739d63 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:01.660015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08b77a-3927-4f08-92f2-e4f3f2739d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:01.660336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08b77a-3927-4f08-92f2-e4f3f2739d63 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:01.661022 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08b77a-3927-4f08-92f2-e4f3f2739d63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:01.663116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca08b77a-3927-4f08-92f2-e4f3f2739d63 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:01.687168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1198/2384] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:30:01 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:30:01.691811 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0face3c4-f1a1-4638-8303-789b649fee6e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:01.703039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0face3c4-f1a1-4638-8303-789b649fee6e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:01.708860 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0723143-6e4d-47fc-822c-c7b13bbca715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:01.709598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0723143-6e4d-47fc-822c-c7b13bbca715 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:01.709921 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0723143-6e4d-47fc-822c-c7b13bbca715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:01.710202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0723143-6e4d-47fc-822c-c7b13bbca715 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:01.710918 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0723143-6e4d-47fc-822c-c7b13bbca715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:01.712592 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0723143-6e4d-47fc-822c-c7b13bbca715 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:01.728599 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1187/2385] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:30:01.729833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08b77a-3927-4f08-92f2-e4f3f2739d63 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:01.742844 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08b77a-3927-4f08-92f2-e4f3f2739d63 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:01.748207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c55dd-6083-496d-a12c-54af9d25dd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:01.748894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c55dd-6083-496d-a12c-54af9d25dd9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:01.749167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c55dd-6083-496d-a12c-54af9d25dd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:01.749438 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c55dd-6083-496d-a12c-54af9d25dd9b admin admin] PATH_INFO: `/v3/projects/ab932a62aad4446c8c85f07038910e89/users/8949e6e0e8134bdfb5c6ffc8037249e5/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:01.767544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c55dd-6083-496d-a12c-54af9d25dd9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab932a62aad4446c8c85f07038910e89, user_id=8949e6e0e8134bdfb5c6ffc8037249e5, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:01.770437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c55dd-6083-496d-a12c-54af9d25dd9b admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:01.787773 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1199/2386] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:30:01 2022] PUT /identity/v3/projects/ab932a62aad4446c8c85f07038910e89/users/8949e6e0e8134bdfb5c6ffc8037249e5/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:01.792315 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0723143-6e4d-47fc-822c-c7b13bbca715 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:01.803226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0723143-6e4d-47fc-822c-c7b13bbca715 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:01.871309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326d738-2c11-4185-8820-25deb5e6e2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:01.872127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326d738-2c11-4185-8820-25deb5e6e2cd admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:01.872273 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326d738-2c11-4185-8820-25deb5e6e2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:01.873199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326d738-2c11-4185-8820-25deb5e6e2cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:01.874094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326d738-2c11-4185-8820-25deb5e6e2cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:01.875998 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326d738-2c11-4185-8820-25deb5e6e2cd admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:01.888705 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1188/2387] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:30:01.893069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c55dd-6083-496d-a12c-54af9d25dd9b admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:01.904652 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c55dd-6083-496d-a12c-54af9d25dd9b admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:01.916154 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da292ec-2d19-4d84-ad22-70f3f0cb46d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:01.916857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da292ec-2d19-4d84-ad22-70f3f0cb46d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:01.917129 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da292ec-2d19-4d84-ad22-70f3f0cb46d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:01.917408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da292ec-2d19-4d84-ad22-70f3f0cb46d3 admin admin] PATH_INFO: `/v3/projects/ab932a62aad4446c8c85f07038910e89/users/8949e6e0e8134bdfb5c6ffc8037249e5/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:01.919671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da292ec-2d19-4d84-ad22-70f3f0cb46d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab932a62aad4446c8c85f07038910e89, user_id=8949e6e0e8134bdfb5c6ffc8037249e5, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:01.922380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da292ec-2d19-4d84-ad22-70f3f0cb46d3 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:01.933711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1200/2388] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:30:01 2022] PUT /identity/v3/projects/ab932a62aad4446c8c85f07038910e89/users/8949e6e0e8134bdfb5c6ffc8037249e5/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:01.949036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d21ee-3593-4be2-bf00-0c6336b2dded None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:01.949348 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d21ee-3593-4be2-bf00-0c6336b2dded None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:01.949627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d21ee-3593-4be2-bf00-0c6336b2dded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:01.979112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d16d21ee-3593-4be2-bf00-0c6336b2dded None None] MFA Rules not processed for user `8949e6e0e8134bdfb5c6ffc8037249e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:01.980355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d16d21ee-3593-4be2-bf00-0c6336b2dded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:02.005742 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:02.057715 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1189/2389] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:30:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:02.071149 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da292ec-2d19-4d84-ad22-70f3f0cb46d3 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:02.073042 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8da292ec-2d19-4d84-ad22-70f3f0cb46d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:02.139975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da292ec-2d19-4d84-ad22-70f3f0cb46d3 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:02.144021 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0362a35-a7b3-44bc-990e-5295ab3d1616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:02.144671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0362a35-a7b3-44bc-990e-5295ab3d1616 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:02.144946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0362a35-a7b3-44bc-990e-5295ab3d1616 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:02.145211 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0362a35-a7b3-44bc-990e-5295ab3d1616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:02.156924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0362a35-a7b3-44bc-990e-5295ab3d1616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:02.160685 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0362a35-a7b3-44bc-990e-5295ab3d1616 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:02.174157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1201/2390] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:30:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:30:02.863791 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d21ee-3593-4be2-bf00-0c6336b2dded None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:02.955740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d21ee-3593-4be2-bf00-0c6336b2dded None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:02.962569 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b307b-30ae-4114-a567-1c8bdccd6a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:02.964004 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b307b-30ae-4114-a567-1c8bdccd6a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:02.964004 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b307b-30ae-4114-a567-1c8bdccd6a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:02.964004 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b307b-30ae-4114-a567-1c8bdccd6a37 admin admin] PATH_INFO: `/v3/users/0c7c239805ee43a698de91e0739f33cd` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:02.965214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b307b-30ae-4114-a567-1c8bdccd6a37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c7c239805ee43a698de91e0739f33cd)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:02.967276 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b307b-30ae-4114-a567-1c8bdccd6a37 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:03.023809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-af2b307b-30ae-4114-a567-1c8bdccd6a37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c7c239805ee43a698de91e0739f33cd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:03.024548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-af2b307b-30ae-4114-a567-1c8bdccd6a37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c7c239805ee43a698de91e0739f33cd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:03.040575 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-af2b307b-30ae-4114-a567-1c8bdccd6a37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c7c239805ee43a698de91e0739f33cd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:03.049099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-af2b307b-30ae-4114-a567-1c8bdccd6a37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c7c239805ee43a698de91e0739f33cd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:03.057278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-af2b307b-30ae-4114-a567-1c8bdccd6a37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c7c239805ee43a698de91e0739f33cd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:03.063751 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1190/2391] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:30:02 2022] DELETE /identity/v3/users/0c7c239805ee43a698de91e0739f33cd => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:30:03.068555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2799e2-9c13-4125-946f-b74e9bc4451b None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:03.069060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2799e2-9c13-4125-946f-b74e9bc4451b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:03.069180 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2799e2-9c13-4125-946f-b74e9bc4451b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:03.093848 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-cf2799e2-9c13-4125-946f-b74e9bc4451b None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:03.095087 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-cf2799e2-9c13-4125-946f-b74e9bc4451b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:03.165805 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1202/2392] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:03.176040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b307b-30ae-4114-a567-1c8bdccd6a37 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:03.177584 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-af2b307b-30ae-4114-a567-1c8bdccd6a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:03.232610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:03.249930 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b307b-30ae-4114-a567-1c8bdccd6a37 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:03.254039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ed64f-09c9-4afd-bdf1-6da52f53071e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:03.254654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ed64f-09c9-4afd-bdf1-6da52f53071e service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:03.254928 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ed64f-09c9-4afd-bdf1-6da52f53071e service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:03.255397 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ed64f-09c9-4afd-bdf1-6da52f53071e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:03.266057 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ed64f-09c9-4afd-bdf1-6da52f53071e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:03.267597 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5ed64f-09c9-4afd-bdf1-6da52f53071e service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:03.279731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1191/2393] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:30:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:30:03.595019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2799e2-9c13-4125-946f-b74e9bc4451b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:03.596605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-cf2799e2-9c13-4125-946f-b74e9bc4451b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:03.612191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2799e2-9c13-4125-946f-b74e9bc4451b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:03.623167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f1ea2-87c2-478a-9a1c-9200547c2c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:03.623978 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1ea2-87c2-478a-9a1c-9200547c2c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:03.624312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1ea2-87c2-478a-9a1c-9200547c2c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:03.624481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1ea2-87c2-478a-9a1c-9200547c2c46 admin admin] PATH_INFO: `/v3/projects/a10dd91271c34beaa1d20d736759c372` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:03.625722 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f1ea2-87c2-478a-9a1c-9200547c2c46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a10dd91271c34beaa1d20d736759c372)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:03.627714 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f1ea2-87c2-478a-9a1c-9200547c2c46 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:03.652935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:03.684283 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-be4f1ea2-87c2-478a-9a1c-9200547c2c46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a10dd91271c34beaa1d20d736759c372', 'request_id': 'req-be4f1ea2-87c2-478a-9a1c-9200547c2c46'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:03.693249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-be4f1ea2-87c2-478a-9a1c-9200547c2c46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a10dd91271c34beaa1d20d736759c372', 'request_id': 'req-be4f1ea2-87c2-478a-9a1c-9200547c2c46'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:03.700179 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-be4f1ea2-87c2-478a-9a1c-9200547c2c46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a10dd91271c34beaa1d20d736759c372', 'request_id': 'req-be4f1ea2-87c2-478a-9a1c-9200547c2c46'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:03.711754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-be4f1ea2-87c2-478a-9a1c-9200547c2c46 admin admin] The token cache is being invalidate because project a10dd91271c34beaa1d20d736759c372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:30:03.712038 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-be4f1ea2-87c2-478a-9a1c-9200547c2c46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:03.712756 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-be4f1ea2-87c2-478a-9a1c-9200547c2c46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:03.714348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1203/2394] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:30:03 2022] DELETE /identity/v3/projects/a10dd91271c34beaa1d20d736759c372 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:03.718929 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ed64f-09c9-4afd-bdf1-6da52f53071e service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:03.730778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5ed64f-09c9-4afd-bdf1-6da52f53071e service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:03.796513 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939eee4-6d00-4342-9bb1-7404a56089f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:03.797164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939eee4-6d00-4342-9bb1-7404a56089f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:03.797441 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939eee4-6d00-4342-9bb1-7404a56089f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:03.797709 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939eee4-6d00-4342-9bb1-7404a56089f0 admin admin] PATH_INFO: `/v3/users/8949e6e0e8134bdfb5c6ffc8037249e5` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:03.798986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939eee4-6d00-4342-9bb1-7404a56089f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8949e6e0e8134bdfb5c6ffc8037249e5)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:03.800971 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939eee4-6d00-4342-9bb1-7404a56089f0 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:03.853229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7939eee4-6d00-4342-9bb1-7404a56089f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8949e6e0e8134bdfb5c6ffc8037249e5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:03.853931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7939eee4-6d00-4342-9bb1-7404a56089f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8949e6e0e8134bdfb5c6ffc8037249e5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:03.871038 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7939eee4-6d00-4342-9bb1-7404a56089f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8949e6e0e8134bdfb5c6ffc8037249e5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:03.882563 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7939eee4-6d00-4342-9bb1-7404a56089f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8949e6e0e8134bdfb5c6ffc8037249e5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:03.891096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7939eee4-6d00-4342-9bb1-7404a56089f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8949e6e0e8134bdfb5c6ffc8037249e5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:03.897753 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1192/2395] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:30:03 2022] DELETE /identity/v3/users/8949e6e0e8134bdfb5c6ffc8037249e5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:04.209323 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f1ea2-87c2-478a-9a1c-9200547c2c46 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:04.211066 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-be4f1ea2-87c2-478a-9a1c-9200547c2c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:04.247561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f1ea2-87c2-478a-9a1c-9200547c2c46 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:04.324757 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79678093-8098-4a7c-a62f-e77f28a99838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:04.325428 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79678093-8098-4a7c-a62f-e77f28a99838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:04.325707 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79678093-8098-4a7c-a62f-e77f28a99838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:04.326034 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79678093-8098-4a7c-a62f-e77f28a99838 admin admin] PATH_INFO: `/v3/projects/ab932a62aad4446c8c85f07038910e89` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:04.327384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79678093-8098-4a7c-a62f-e77f28a99838 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab932a62aad4446c8c85f07038910e89)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:04.329612 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79678093-8098-4a7c-a62f-e77f28a99838 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:04.382660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-79678093-8098-4a7c-a62f-e77f28a99838 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab932a62aad4446c8c85f07038910e89', 'request_id': 'req-79678093-8098-4a7c-a62f-e77f28a99838'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:04.393034 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-79678093-8098-4a7c-a62f-e77f28a99838 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab932a62aad4446c8c85f07038910e89', 'request_id': 'req-79678093-8098-4a7c-a62f-e77f28a99838'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:04.400303 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-79678093-8098-4a7c-a62f-e77f28a99838 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab932a62aad4446c8c85f07038910e89', 'request_id': 'req-79678093-8098-4a7c-a62f-e77f28a99838'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:04.411727 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-79678093-8098-4a7c-a62f-e77f28a99838 admin admin] The token cache is being invalidate because project ab932a62aad4446c8c85f07038910e89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:30:04.412015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-79678093-8098-4a7c-a62f-e77f28a99838 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:04.412701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-79678093-8098-4a7c-a62f-e77f28a99838 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:04.414335 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1204/2396] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:30:04 2022] DELETE /identity/v3/projects/ab932a62aad4446c8c85f07038910e89 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:04.421417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeffff6-d062-4ed4-b86a-a09c251ae8b5 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:04.421974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeffff6-d062-4ed4-b86a-a09c251ae8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:04.422088 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeffff6-d062-4ed4-b86a-a09c251ae8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:04.443845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-deeffff6-d062-4ed4-b86a-a09c251ae8b5 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:04.445085 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-deeffff6-d062-4ed4-b86a-a09c251ae8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:04.511970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1193/2397] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:04.585749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cf7ef-abe6-4071-b729-391c922ea20c None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:04.586071 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cf7ef-abe6-4071-b729-391c922ea20c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:04.586206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3cf7ef-abe6-4071-b729-391c922ea20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:04.608047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2b3cf7ef-abe6-4071-b729-391c922ea20c None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:04.609276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2b3cf7ef-abe6-4071-b729-391c922ea20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:04.618979 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1205/2398] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:04.624023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeffff6-d062-4ed4-b86a-a09c251ae8b5 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:04.630644 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:04.637199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeffff6-d062-4ed4-b86a-a09c251ae8b5 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:04.644560 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df53cb-17c4-49a8-89de-c3c9bb13a927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:04.645202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df53cb-17c4-49a8-89de-c3c9bb13a927 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:04.645492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df53cb-17c4-49a8-89de-c3c9bb13a927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:04.645831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df53cb-17c4-49a8-89de-c3c9bb13a927 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:04.646786 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df53cb-17c4-49a8-89de-c3c9bb13a927 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:30:04.647095 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df53cb-17c4-49a8-89de-c3c9bb13a927 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:04.648805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df53cb-17c4-49a8-89de-c3c9bb13a927 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:04.661003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1194/2399] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:30:04.667158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cf7ef-abe6-4071-b729-391c922ea20c None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:04.677261 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3cf7ef-abe6-4071-b729-391c922ea20c None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:04.683422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80a6b7-d62e-4c99-98c4-ab551fd74432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:04.684073 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80a6b7-d62e-4c99-98c4-ab551fd74432 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:04.684382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80a6b7-d62e-4c99-98c4-ab551fd74432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:04.684639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80a6b7-d62e-4c99-98c4-ab551fd74432 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:04.685397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80a6b7-d62e-4c99-98c4-ab551fd74432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:04.687382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e80a6b7-d62e-4c99-98c4-ab551fd74432 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:04.721888 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1206/2400] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:30:04 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:30:04.726541 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df53cb-17c4-49a8-89de-c3c9bb13a927 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:04.736937 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df53cb-17c4-49a8-89de-c3c9bb13a927 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:04.792845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611131d9-78ff-41d4-8da4-647812a49745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:04.793466 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611131d9-78ff-41d4-8da4-647812a49745 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:04.793748 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611131d9-78ff-41d4-8da4-647812a49745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:04.794091 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611131d9-78ff-41d4-8da4-647812a49745 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:04.794748 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611131d9-78ff-41d4-8da4-647812a49745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:04.796702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611131d9-78ff-41d4-8da4-647812a49745 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:04.820027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1195/2401] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:30:04 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:30:04.826103 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80a6b7-d62e-4c99-98c4-ab551fd74432 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:04.837478 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80a6b7-d62e-4c99-98c4-ab551fd74432 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:04.843626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5cefb-47ce-4f30-83e1-84598f469095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:04.844312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5cefb-47ce-4f30-83e1-84598f469095 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:04.844590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5cefb-47ce-4f30-83e1-84598f469095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:04.844856 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5cefb-47ce-4f30-83e1-84598f469095 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:04.845565 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5cefb-47ce-4f30-83e1-84598f469095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:04.847220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5cefb-47ce-4f30-83e1-84598f469095 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:04.852872 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:04.858555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1207/2402] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:30:04.862876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611131d9-78ff-41d4-8da4-647812a49745 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:04.874116 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611131d9-78ff-41d4-8da4-647812a49745 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:04.880140 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73acd396-056e-4050-b15d-c696ec485f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:04.880807 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73acd396-056e-4050-b15d-c696ec485f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:04.881087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73acd396-056e-4050-b15d-c696ec485f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:04.881359 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73acd396-056e-4050-b15d-c696ec485f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:04.882132 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73acd396-056e-4050-b15d-c696ec485f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:04.883754 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73acd396-056e-4050-b15d-c696ec485f07 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:04.895484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1196/2403] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:30:04.899970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5cefb-47ce-4f30-83e1-84598f469095 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:04.910604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5cefb-47ce-4f30-83e1-84598f469095 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:04.915777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc7f7f-c75f-4af2-a752-044f08d10852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:04.916445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc7f7f-c75f-4af2-a752-044f08d10852 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:04.916724 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc7f7f-c75f-4af2-a752-044f08d10852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:04.916993 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc7f7f-c75f-4af2-a752-044f08d10852 admin admin] PATH_INFO: `/v3/projects/44c75fa536b447eb9d9f4512dc3153b9/users/d1203cb44cb8441b8bed62a5635c823e/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:04.932493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc7f7f-c75f-4af2-a752-044f08d10852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c75fa536b447eb9d9f4512dc3153b9, user_id=d1203cb44cb8441b8bed62a5635c823e, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:04.935295 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc7f7f-c75f-4af2-a752-044f08d10852 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:04.950224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1208/2404] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:30:04 2022] PUT /identity/v3/projects/44c75fa536b447eb9d9f4512dc3153b9/users/d1203cb44cb8441b8bed62a5635c823e/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:04.960961 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccce822-bb58-4396-b025-80416cc3066e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:04.961252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccce822-bb58-4396-b025-80416cc3066e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:04.961515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccce822-bb58-4396-b025-80416cc3066e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:04.994084 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6ccce822-bb58-4396-b025-80416cc3066e None None] MFA Rules not processed for user `d1203cb44cb8441b8bed62a5635c823e`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:04.995328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6ccce822-bb58-4396-b025-80416cc3066e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:05.073139 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1197/2405] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:30:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:05.087860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc7f7f-c75f-4af2-a752-044f08d10852 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:05.089949 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-72fc7f7f-c75f-4af2-a752-044f08d10852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:05.165451 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc7f7f-c75f-4af2-a752-044f08d10852 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:05.169448 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277364ee-8f7c-4afc-91f6-b06b10aa1556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:05.170108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277364ee-8f7c-4afc-91f6-b06b10aa1556 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:05.170393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277364ee-8f7c-4afc-91f6-b06b10aa1556 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:05.170670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277364ee-8f7c-4afc-91f6-b06b10aa1556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:05.180193 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277364ee-8f7c-4afc-91f6-b06b10aa1556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:05.182390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277364ee-8f7c-4afc-91f6-b06b10aa1556 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:05.195960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1209/2406] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:30:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:30:05.222187 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccce822-bb58-4396-b025-80416cc3066e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:05.285816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:05.290858 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccce822-bb58-4396-b025-80416cc3066e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:05.298502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c3a45-89d8-4ac0-a28c-b9cbbbdfdda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:05.299436 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c3a45-89d8-4ac0-a28c-b9cbbbdfdda0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:05.299774 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c3a45-89d8-4ac0-a28c-b9cbbbdfdda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:05.300137 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c3a45-89d8-4ac0-a28c-b9cbbbdfdda0 admin admin] PATH_INFO: `/v3/users/d1203cb44cb8441b8bed62a5635c823e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:05.301876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c3a45-89d8-4ac0-a28c-b9cbbbdfdda0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1203cb44cb8441b8bed62a5635c823e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:05.304739 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c3a45-89d8-4ac0-a28c-b9cbbbdfdda0 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:05.366509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-754c3a45-89d8-4ac0-a28c-b9cbbbdfdda0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1203cb44cb8441b8bed62a5635c823e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:05.367221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-754c3a45-89d8-4ac0-a28c-b9cbbbdfdda0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1203cb44cb8441b8bed62a5635c823e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:05.379628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-754c3a45-89d8-4ac0-a28c-b9cbbbdfdda0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1203cb44cb8441b8bed62a5635c823e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:05.390273 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-754c3a45-89d8-4ac0-a28c-b9cbbbdfdda0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1203cb44cb8441b8bed62a5635c823e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:05.398376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-754c3a45-89d8-4ac0-a28c-b9cbbbdfdda0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1203cb44cb8441b8bed62a5635c823e'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:05.405091 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1198/2407] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:30:05 2022] DELETE /identity/v3/users/d1203cb44cb8441b8bed62a5635c823e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:05.409905 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af81ce-20b8-43f4-a223-7e2945e95503 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:05.410418 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af81ce-20b8-43f4-a223-7e2945e95503 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:05.410418 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af81ce-20b8-43f4-a223-7e2945e95503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:05.431002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-77af81ce-20b8-43f4-a223-7e2945e95503 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:05.432235 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-77af81ce-20b8-43f4-a223-7e2945e95503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:05.507184 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1210/2408] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:05.519012 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c3a45-89d8-4ac0-a28c-b9cbbbdfdda0 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:05.520533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-754c3a45-89d8-4ac0-a28c-b9cbbbdfdda0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:05.583816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c3a45-89d8-4ac0-a28c-b9cbbbdfdda0 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:05.589397 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e7348-94c4-4e27-85ec-bcd8a6959c8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:05.590046 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e7348-94c4-4e27-85ec-bcd8a6959c8c service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:05.590328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e7348-94c4-4e27-85ec-bcd8a6959c8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:05.590616 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e7348-94c4-4e27-85ec-bcd8a6959c8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:05.601772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e7348-94c4-4e27-85ec-bcd8a6959c8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:05.603210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e7348-94c4-4e27-85ec-bcd8a6959c8c service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:05.613730 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1199/2409] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:30:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:30:05.902601 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af81ce-20b8-43f4-a223-7e2945e95503 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:05.904092 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-77af81ce-20b8-43f4-a223-7e2945e95503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:05.916689 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af81ce-20b8-43f4-a223-7e2945e95503 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:05.922822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcd7d3-b903-4e19-8d75-a6354fd6bd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:05.923519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcd7d3-b903-4e19-8d75-a6354fd6bd5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:05.923801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcd7d3-b903-4e19-8d75-a6354fd6bd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:05.924053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcd7d3-b903-4e19-8d75-a6354fd6bd5d admin admin] PATH_INFO: `/v3/projects/44c75fa536b447eb9d9f4512dc3153b9` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:05.926644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcd7d3-b903-4e19-8d75-a6354fd6bd5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=44c75fa536b447eb9d9f4512dc3153b9)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:05.928850 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcd7d3-b903-4e19-8d75-a6354fd6bd5d admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:05.979957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-05fcd7d3-b903-4e19-8d75-a6354fd6bd5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44c75fa536b447eb9d9f4512dc3153b9', 'request_id': 'req-05fcd7d3-b903-4e19-8d75-a6354fd6bd5d'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:05.988488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-05fcd7d3-b903-4e19-8d75-a6354fd6bd5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44c75fa536b447eb9d9f4512dc3153b9', 'request_id': 'req-05fcd7d3-b903-4e19-8d75-a6354fd6bd5d'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:05.995586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-05fcd7d3-b903-4e19-8d75-a6354fd6bd5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44c75fa536b447eb9d9f4512dc3153b9', 'request_id': 'req-05fcd7d3-b903-4e19-8d75-a6354fd6bd5d'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:06.005912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-05fcd7d3-b903-4e19-8d75-a6354fd6bd5d admin admin] The token cache is being invalidate because project 44c75fa536b447eb9d9f4512dc3153b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:30:06.006198 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-05fcd7d3-b903-4e19-8d75-a6354fd6bd5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:06.006878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-05fcd7d3-b903-4e19-8d75-a6354fd6bd5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:06.008462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1211/2410] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:30:05 2022] DELETE /identity/v3/projects/44c75fa536b447eb9d9f4512dc3153b9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:06.013812 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02787806-4000-4bac-90b7-eb27e0a64883 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:06.014122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02787806-4000-4bac-90b7-eb27e0a64883 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:06.014413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02787806-4000-4bac-90b7-eb27e0a64883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:06.035282 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-02787806-4000-4bac-90b7-eb27e0a64883 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:06.036604 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-02787806-4000-4bac-90b7-eb27e0a64883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:06.107247 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1200/2411] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:30:06.115563 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061a104-f5e2-46d1-b3eb-be74031b8568 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:06.115863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061a104-f5e2-46d1-b3eb-be74031b8568 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:06.116146 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061a104-f5e2-46d1-b3eb-be74031b8568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:06.124359 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:06.139761 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-d061a104-f5e2-46d1-b3eb-be74031b8568 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:06.140922 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d061a104-f5e2-46d1-b3eb-be74031b8568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:06.149249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1212/2412] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:30:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:06.153747 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02787806-4000-4bac-90b7-eb27e0a64883 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:06.164222 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02787806-4000-4bac-90b7-eb27e0a64883 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:06.170162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112fafc8-706e-4625-a30e-dc6134c8a4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:06.170802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fafc8-706e-4625-a30e-dc6134c8a4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:06.171079 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fafc8-706e-4625-a30e-dc6134c8a4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:06.171349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112fafc8-706e-4625-a30e-dc6134c8a4a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:06.172087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112fafc8-706e-4625-a30e-dc6134c8a4a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:30:06.172389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112fafc8-706e-4625-a30e-dc6134c8a4a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:06.174060 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112fafc8-706e-4625-a30e-dc6134c8a4a2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:06.186560 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1201/2413] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:30:06.191564 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061a104-f5e2-46d1-b3eb-be74031b8568 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:06.201524 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061a104-f5e2-46d1-b3eb-be74031b8568 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:06.208029 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2009b8-72bd-4b17-a864-d3fec6cead1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:06.208645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2009b8-72bd-4b17-a864-d3fec6cead1e admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:06.208922 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2009b8-72bd-4b17-a864-d3fec6cead1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:06.209191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2009b8-72bd-4b17-a864-d3fec6cead1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:06.209958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2009b8-72bd-4b17-a864-d3fec6cead1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:06.211899 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2009b8-72bd-4b17-a864-d3fec6cead1e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:06.244936 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1213/2414] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:30:06 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:30:06.250045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112fafc8-706e-4625-a30e-dc6134c8a4a2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:06.263869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112fafc8-706e-4625-a30e-dc6134c8a4a2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:06.268174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:06.325357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05c1ba-df57-4017-9475-bf5d2251e63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:06.326032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05c1ba-df57-4017-9475-bf5d2251e63f admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:06.326311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05c1ba-df57-4017-9475-bf5d2251e63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:06.326567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05c1ba-df57-4017-9475-bf5d2251e63f admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:06.327242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05c1ba-df57-4017-9475-bf5d2251e63f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:06.329190 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05c1ba-df57-4017-9475-bf5d2251e63f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:06.353983 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1202/2415] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:30:06 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:30:06.360028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2009b8-72bd-4b17-a864-d3fec6cead1e admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:06.373657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2009b8-72bd-4b17-a864-d3fec6cead1e admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:06.378925 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328122f-c58c-45c2-ae21-cbc6b5567b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:06.379540 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328122f-c58c-45c2-ae21-cbc6b5567b89 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:06.379813 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328122f-c58c-45c2-ae21-cbc6b5567b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:06.380086 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328122f-c58c-45c2-ae21-cbc6b5567b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:06.380793 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328122f-c58c-45c2-ae21-cbc6b5567b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:06.382508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328122f-c58c-45c2-ae21-cbc6b5567b89 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:06.393562 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1214/2416] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:30:06.398869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05c1ba-df57-4017-9475-bf5d2251e63f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:06.409970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05c1ba-df57-4017-9475-bf5d2251e63f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:06.415087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e43cf-2461-4598-a5e9-70d5472cb4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:06.415698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e43cf-2461-4598-a5e9-70d5472cb4fe admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:06.415974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e43cf-2461-4598-a5e9-70d5472cb4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:06.416243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e43cf-2461-4598-a5e9-70d5472cb4fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:06.417028 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e43cf-2461-4598-a5e9-70d5472cb4fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:06.418704 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e43cf-2461-4598-a5e9-70d5472cb4fe admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:06.430598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1203/2417] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:30:06.435159 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328122f-c58c-45c2-ae21-cbc6b5567b89 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:06.446338 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328122f-c58c-45c2-ae21-cbc6b5567b89 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:06.452025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a8352-6c9f-47d6-9402-5b4dfecc4396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:06.452699 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a8352-6c9f-47d6-9402-5b4dfecc4396 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:06.452976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a8352-6c9f-47d6-9402-5b4dfecc4396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:06.453248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a8352-6c9f-47d6-9402-5b4dfecc4396 admin admin] PATH_INFO: `/v3/projects/a88c63b5f05843178c9000e7a42c1b39/users/8460813c148245b59370ce03cc91b61f/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:06.469081 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a8352-6c9f-47d6-9402-5b4dfecc4396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a88c63b5f05843178c9000e7a42c1b39, user_id=8460813c148245b59370ce03cc91b61f, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:06.471795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a8352-6c9f-47d6-9402-5b4dfecc4396 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:06.487848 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1215/2418] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:30:06 2022] PUT /identity/v3/projects/a88c63b5f05843178c9000e7a42c1b39/users/8460813c148245b59370ce03cc91b61f/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:30:06.500803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eee3a6-ab13-4a8b-844a-6bf24a6ee025 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:06.501102 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eee3a6-ab13-4a8b-844a-6bf24a6ee025 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:06.501377 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eee3a6-ab13-4a8b-844a-6bf24a6ee025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:06.538209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-98eee3a6-ab13-4a8b-844a-6bf24a6ee025 None None] MFA Rules not processed for user `8460813c148245b59370ce03cc91b61f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:06.539490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-98eee3a6-ab13-4a8b-844a-6bf24a6ee025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:06.607829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1204/2419] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:30:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:06.612631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a8352-6c9f-47d6-9402-5b4dfecc4396 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:06.681984 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a8352-6c9f-47d6-9402-5b4dfecc4396 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:06.687082 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c9174-4fbe-41be-aa2f-0472de64c43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:06.687785 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c9174-4fbe-41be-aa2f-0472de64c43f admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:06.688313 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c9174-4fbe-41be-aa2f-0472de64c43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:06.688656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c9174-4fbe-41be-aa2f-0472de64c43f admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:06.689396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c9174-4fbe-41be-aa2f-0472de64c43f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:06.691582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4c9174-4fbe-41be-aa2f-0472de64c43f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:06.727155 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1216/2420] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:30:06 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:30:06.728405 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eee3a6-ab13-4a8b-844a-6bf24a6ee025 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:06.739509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eee3a6-ab13-4a8b-844a-6bf24a6ee025 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:06.798074 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b5012-3e97-48fa-86ba-d2a06c3170a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:06.798700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b5012-3e97-48fa-86ba-d2a06c3170a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:06.798982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b5012-3e97-48fa-86ba-d2a06c3170a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:06.799257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b5012-3e97-48fa-86ba-d2a06c3170a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:06.799921 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b5012-3e97-48fa-86ba-d2a06c3170a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:06.801965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b5012-3e97-48fa-86ba-d2a06c3170a9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:06.826283 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1205/2421] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:30:06 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:30:06.830384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c9174-4fbe-41be-aa2f-0472de64c43f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:06.841580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4c9174-4fbe-41be-aa2f-0472de64c43f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:06.847259 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee58c58-57c4-4d8f-9b95-836999c08dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:06.847880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee58c58-57c4-4d8f-9b95-836999c08dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:06.848156 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee58c58-57c4-4d8f-9b95-836999c08dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:06.848484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee58c58-57c4-4d8f-9b95-836999c08dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:06.849135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee58c58-57c4-4d8f-9b95-836999c08dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:06.850862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee58c58-57c4-4d8f-9b95-836999c08dfb admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:06.862005 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1217/2422] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:30:06.866664 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b5012-3e97-48fa-86ba-d2a06c3170a9 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:06.874876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:06.879426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b5012-3e97-48fa-86ba-d2a06c3170a9 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:06.886283 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cee5ff-01e8-455a-99f7-54cb299085b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:06.886974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cee5ff-01e8-455a-99f7-54cb299085b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:06.887250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cee5ff-01e8-455a-99f7-54cb299085b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:06.887549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cee5ff-01e8-455a-99f7-54cb299085b7 admin admin] PATH_INFO: `/v3/projects/734e210c66b943b7b18c0e31375fc7e3/users/7652b7814c0c4b39b91e8d25901453ef/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:06.904469 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cee5ff-01e8-455a-99f7-54cb299085b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=734e210c66b943b7b18c0e31375fc7e3, user_id=7652b7814c0c4b39b91e8d25901453ef, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:06.907230 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cee5ff-01e8-455a-99f7-54cb299085b7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:06.921895 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1206/2423] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:30:06 2022] PUT /identity/v3/projects/734e210c66b943b7b18c0e31375fc7e3/users/7652b7814c0c4b39b91e8d25901453ef/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:06.926313 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee58c58-57c4-4d8f-9b95-836999c08dfb admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:06.936820 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee58c58-57c4-4d8f-9b95-836999c08dfb admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:06.994809 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce93ec2-41d9-4a01-9240-d1a4e98cdf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:06.995427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce93ec2-41d9-4a01-9240-d1a4e98cdf05 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:06.995707 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce93ec2-41d9-4a01-9240-d1a4e98cdf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:06.995981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce93ec2-41d9-4a01-9240-d1a4e98cdf05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:06.996687 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce93ec2-41d9-4a01-9240-d1a4e98cdf05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:06.998411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce93ec2-41d9-4a01-9240-d1a4e98cdf05 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:07.008633 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1218/2424] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:30:07.013145 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cee5ff-01e8-455a-99f7-54cb299085b7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:07.023734 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cee5ff-01e8-455a-99f7-54cb299085b7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:07.028904 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9373ba-2633-45ed-8f86-08a8e617de14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:07.029580 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9373ba-2633-45ed-8f86-08a8e617de14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:07.029893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9373ba-2633-45ed-8f86-08a8e617de14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:07.030193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9373ba-2633-45ed-8f86-08a8e617de14 admin admin] PATH_INFO: `/v3/projects/734e210c66b943b7b18c0e31375fc7e3/users/7652b7814c0c4b39b91e8d25901453ef/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:07.032305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9373ba-2633-45ed-8f86-08a8e617de14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=734e210c66b943b7b18c0e31375fc7e3, user_id=7652b7814c0c4b39b91e8d25901453ef, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:07.035001 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9373ba-2633-45ed-8f86-08a8e617de14 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:07.045483 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1207/2425] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:30:07 2022] PUT /identity/v3/projects/734e210c66b943b7b18c0e31375fc7e3/users/7652b7814c0c4b39b91e8d25901453ef/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:07.055899 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e6f7f-1607-4f59-accd-0344c325714c None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:07.056193 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e6f7f-1607-4f59-accd-0344c325714c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:07.056463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e6f7f-1607-4f59-accd-0344c325714c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:07.076356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:07.087749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-351e6f7f-1607-4f59-accd-0344c325714c None None] MFA Rules not processed for user `7652b7814c0c4b39b91e8d25901453ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:07.089025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-351e6f7f-1607-4f59-accd-0344c325714c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:07.181309 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1219/2426] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:30:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:07.192740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9373ba-2633-45ed-8f86-08a8e617de14 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:07.194622 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ea9373ba-2633-45ed-8f86-08a8e617de14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:07.261269 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9373ba-2633-45ed-8f86-08a8e617de14 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:07.265788 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b91356-4a67-4f7b-8182-a3ceab05e886 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:07.266421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b91356-4a67-4f7b-8182-a3ceab05e886 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:07.266693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b91356-4a67-4f7b-8182-a3ceab05e886 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:07.266963 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b91356-4a67-4f7b-8182-a3ceab05e886 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:07.327291 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b91356-4a67-4f7b-8182-a3ceab05e886 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:07.328757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b91356-4a67-4f7b-8182-a3ceab05e886 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:07.338877 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1208/2427] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:30:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:30:09.911999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e6f7f-1607-4f59-accd-0344c325714c None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:09.979162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e6f7f-1607-4f59-accd-0344c325714c None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:09.984074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d75cf-7505-4eb4-9f7b-d26ec783132b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:09.984727 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d75cf-7505-4eb4-9f7b-d26ec783132b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:09.985007 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d75cf-7505-4eb4-9f7b-d26ec783132b admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:09.985281 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d75cf-7505-4eb4-9f7b-d26ec783132b admin admin] PATH_INFO: `/v3/users/8460813c148245b59370ce03cc91b61f` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:09.986496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d75cf-7505-4eb4-9f7b-d26ec783132b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8460813c148245b59370ce03cc91b61f)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:09.988446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d75cf-7505-4eb4-9f7b-d26ec783132b admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:10.039247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-276d75cf-7505-4eb4-9f7b-d26ec783132b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8460813c148245b59370ce03cc91b61f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:10.047835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-276d75cf-7505-4eb4-9f7b-d26ec783132b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8460813c148245b59370ce03cc91b61f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:10.048505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-276d75cf-7505-4eb4-9f7b-d26ec783132b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8460813c148245b59370ce03cc91b61f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:10.055617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-276d75cf-7505-4eb4-9f7b-d26ec783132b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8460813c148245b59370ce03cc91b61f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:10.067441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-276d75cf-7505-4eb4-9f7b-d26ec783132b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8460813c148245b59370ce03cc91b61f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:10.073354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1220/2428] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:30:09 2022] DELETE /identity/v3/users/8460813c148245b59370ce03cc91b61f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:10.078034 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57201ff6-4f72-4359-8865-9089a405788a None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:10.078565 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57201ff6-4f72-4359-8865-9089a405788a None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:10.078565 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57201ff6-4f72-4359-8865-9089a405788a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:10.101507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-57201ff6-4f72-4359-8865-9089a405788a None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:10.102810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-57201ff6-4f72-4359-8865-9089a405788a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:10.171083 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1209/2429] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:30:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:10.182962 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d75cf-7505-4eb4-9f7b-d26ec783132b admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:10.184616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-276d75cf-7505-4eb4-9f7b-d26ec783132b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:10.200147 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:10.252731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d75cf-7505-4eb4-9f7b-d26ec783132b admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:10.256778 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2859ea9-36a8-43dd-a4b1-309af0957ed6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:10.257410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2859ea9-36a8-43dd-a4b1-309af0957ed6 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:10.257680 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2859ea9-36a8-43dd-a4b1-309af0957ed6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:10.257983 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2859ea9-36a8-43dd-a4b1-309af0957ed6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:10.268300 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2859ea9-36a8-43dd-a4b1-309af0957ed6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:10.269810 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2859ea9-36a8-43dd-a4b1-309af0957ed6 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:10.280445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1221/2430] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:30:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:30:10.585264 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57201ff6-4f72-4359-8865-9089a405788a None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:10.586818 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-57201ff6-4f72-4359-8865-9089a405788a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:10.602501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57201ff6-4f72-4359-8865-9089a405788a None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:10.609590 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:10.610742 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:10.611202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:10.611690 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39 admin admin] PATH_INFO: `/v3/projects/a88c63b5f05843178c9000e7a42c1b39` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:10.613678 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a88c63b5f05843178c9000e7a42c1b39)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:10.616243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:10.636781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:10.670504 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a88c63b5f05843178c9000e7a42c1b39', 'request_id': 'req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:10.676496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a88c63b5f05843178c9000e7a42c1b39', 'request_id': 'req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:10.685636 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a88c63b5f05843178c9000e7a42c1b39', 'request_id': 'req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:10.698728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39 admin admin] The token cache is being invalidate because project a88c63b5f05843178c9000e7a42c1b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:30:10.699213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:10.700282 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:10.702782 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1210/2431] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:30:10 2022] DELETE /identity/v3/projects/a88c63b5f05843178c9000e7a42c1b39 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:10.707075 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2859ea9-36a8-43dd-a4b1-309af0957ed6 service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:10.718280 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2859ea9-36a8-43dd-a4b1-309af0957ed6 service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:10.799373 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2662346-7c56-4328-858d-e54f13a89951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:10.800039 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2662346-7c56-4328-858d-e54f13a89951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:10.800314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2662346-7c56-4328-858d-e54f13a89951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:10.800614 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2662346-7c56-4328-858d-e54f13a89951 admin admin] PATH_INFO: `/v3/users/7652b7814c0c4b39b91e8d25901453ef` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:10.801906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2662346-7c56-4328-858d-e54f13a89951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7652b7814c0c4b39b91e8d25901453ef)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:10.803902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2662346-7c56-4328-858d-e54f13a89951 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:10.856591 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c2662346-7c56-4328-858d-e54f13a89951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7652b7814c0c4b39b91e8d25901453ef'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:10.868955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c2662346-7c56-4328-858d-e54f13a89951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7652b7814c0c4b39b91e8d25901453ef'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:10.869685 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c2662346-7c56-4328-858d-e54f13a89951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7652b7814c0c4b39b91e8d25901453ef'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:10.881547 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c2662346-7c56-4328-858d-e54f13a89951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7652b7814c0c4b39b91e8d25901453ef'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:10.896519 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c2662346-7c56-4328-858d-e54f13a89951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7652b7814c0c4b39b91e8d25901453ef'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:10.902670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1222/2432] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:30:10 2022] DELETE /identity/v3/users/7652b7814c0c4b39b91e8d25901453ef => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:11.196701 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:11.198418 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:11.215568 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07978a-80ae-4cfc-aaa3-18d0e009ef39 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:11.288970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:11.289753 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:11.290178 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:11.290463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2 admin admin] PATH_INFO: `/v3/projects/734e210c66b943b7b18c0e31375fc7e3` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:11.291709 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=734e210c66b943b7b18c0e31375fc7e3)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:11.293728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:11.348099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '734e210c66b943b7b18c0e31375fc7e3', 'request_id': 'req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:11.355076 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '734e210c66b943b7b18c0e31375fc7e3', 'request_id': 'req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:11.363430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '734e210c66b943b7b18c0e31375fc7e3', 'request_id': 'req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:11.375641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2 admin admin] The token cache is being invalidate because project 734e210c66b943b7b18c0e31375fc7e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:30:11.375932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:11.376589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:30:11.378385 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1211/2433] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:30:11 2022] DELETE /identity/v3/projects/734e210c66b943b7b18c0e31375fc7e3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:30:18.951069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6710e99-4d19-41da-8c75-0e70b4f57221 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:18.951581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6710e99-4d19-41da-8c75-0e70b4f57221 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:18.951672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6710e99-4d19-41da-8c75-0e70b4f57221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:18.971231 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-f6710e99-4d19-41da-8c75-0e70b4f57221 None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:18.972452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f6710e99-4d19-41da-8c75-0e70b4f57221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:19.003429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:19.039308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1223/2434] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 15:30:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:19.102157 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:19.104283 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:19.155492 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:19.201543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f86f9-4ee6-4f65-98c2-e899ad5b41d2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:19.209935 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b469b9c-6c18-4586-b9e4-eb713238ed42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:19.210639 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b469b9c-6c18-4586-b9e4-eb713238ed42 service swift] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:19.211093 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b469b9c-6c18-4586-b9e4-eb713238ed42 service swift] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:19.211369 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b469b9c-6c18-4586-b9e4-eb713238ed42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:19.224610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b469b9c-6c18-4586-b9e4-eb713238ed42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:19.226232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b469b9c-6c18-4586-b9e4-eb713238ed42 service swift] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:19.238299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1212/2435] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:30:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:30:29.028406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=9 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:29.233337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=16 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:33.259213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0417a-1014-4966-b2ad-f66c8a0cd2bd None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:33.259828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0417a-1014-4966-b2ad-f66c8a0cd2bd None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:33.259828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0417a-1014-4966-b2ad-f66c8a0cd2bd None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:33.261364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1224/2436] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 15:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:30:33.268375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf1aff-24fd-495d-b201-3be926837e37 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:33.269366 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf1aff-24fd-495d-b201-3be926837e37 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:33.269683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf1aff-24fd-495d-b201-3be926837e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:33.291538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-adaf1aff-24fd-495d-b201-3be926837e37 None None] MFA Rules not processed for user `2de0ff99a2b74979a28a8b10e9ebdfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:33.292790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-adaf1aff-24fd-495d-b201-3be926837e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:33.369449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1213/2437] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 15:30:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:33.381998 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0417a-1014-4966-b2ad-f66c8a0cd2bd None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:33.384454 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-bfd0417a-1014-4966-b2ad-f66c8a0cd2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:33.465406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0417a-1014-4966-b2ad-f66c8a0cd2bd None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:33.470560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae1ef6-1fc7-4435-a773-2fdbaa6e4625 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f696dfe4b34c4aa19104b5f7ee6e70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:33.471193 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae1ef6-1fc7-4435-a773-2fdbaa6e4625 service glance] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:33.471470 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae1ef6-1fc7-4435-a773-2fdbaa6e4625 service glance] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:33.471737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae1ef6-1fc7-4435-a773-2fdbaa6e4625 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:33.484205 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae1ef6-1fc7-4435-a773-2fdbaa6e4625 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:33.485660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae1ef6-1fc7-4435-a773-2fdbaa6e4625 service glance] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:33.497137 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1225/2438] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:30:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:30:33.572818 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1acb6-52ac-4944-89ae-983f3a3e0ec1 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:33.573136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1acb6-52ac-4944-89ae-983f3a3e0ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:33.573336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1acb6-52ac-4944-89ae-983f3a3e0ec1 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:33.574746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1214/2439] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 15:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:30:33.579670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21537d59-2e4b-445b-9106-ace9e9b9982d None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:33.579992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21537d59-2e4b-445b-9106-ace9e9b9982d None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:33.580722 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21537d59-2e4b-445b-9106-ace9e9b9982d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:33.602678 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-21537d59-2e4b-445b-9106-ace9e9b9982d None None] MFA Rules not processed for user `5ef6577b10f7426a925d594e627f27cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:33.603943 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-21537d59-2e4b-445b-9106-ace9e9b9982d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:33.668908 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1226/2440] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 15:30:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:30:33.681623 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1acb6-52ac-4944-89ae-983f3a3e0ec1 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:33.683575 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cce1acb6-52ac-4944-89ae-983f3a3e0ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:33.697974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1acb6-52ac-4944-89ae-983f3a3e0ec1 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:33.701961 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744a7e9-d3a9-438d-8f71-da95f22c0636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:33.702601 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744a7e9-d3a9-438d-8f71-da95f22c0636 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:33.702882 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744a7e9-d3a9-438d-8f71-da95f22c0636 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:33.703153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744a7e9-d3a9-438d-8f71-da95f22c0636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:33.711599 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744a7e9-d3a9-438d-8f71-da95f22c0636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:33.713082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744a7e9-d3a9-438d-8f71-da95f22c0636 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:33.721939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1215/2441] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:30:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:30:33.807946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e366b49-75a3-4e7d-9b5a-a8e6a51f8840 None None] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:33.808203 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e366b49-75a3-4e7d-9b5a-a8e6a51f8840 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:33.808449 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e366b49-75a3-4e7d-9b5a-a8e6a51f8840 None None] PATH_INFO: `/` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:33.810138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1227/2442] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 15:30:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:30:33.819793 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473f750-3061-422a-80d6-508f2c7c2692 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:33.820253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473f750-3061-422a-80d6-508f2c7c2692 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:33.820731 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473f750-3061-422a-80d6-508f2c7c2692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:33.843877 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e473f750-3061-422a-80d6-508f2c7c2692 None None] MFA Rules not processed for user `2de0ff99a2b74979a28a8b10e9ebdfcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:33.845127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e473f750-3061-422a-80d6-508f2c7c2692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:33.858253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1216/2443] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 15:30:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:33.870988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e366b49-75a3-4e7d-9b5a-a8e6a51f8840 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:33.886914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e366b49-75a3-4e7d-9b5a-a8e6a51f8840 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:33.891076 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5da760-8f36-4e79-97fe-5ad23116746f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:33.891713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5da760-8f36-4e79-97fe-5ad23116746f service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:33.891987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5da760-8f36-4e79-97fe-5ad23116746f service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:33.892253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5da760-8f36-4e79-97fe-5ad23116746f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:33.905077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5da760-8f36-4e79-97fe-5ad23116746f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:33.906698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5da760-8f36-4e79-97fe-5ad23116746f service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:33.919318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1228/2444] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:30:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:30:37.402552 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2856de3-d0ad-4d35-bc41-c0989ab6615e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:37.403070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2856de3-d0ad-4d35-bc41-c0989ab6615e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:37.403185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2856de3-d0ad-4d35-bc41-c0989ab6615e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:37.424349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f2856de3-d0ad-4d35-bc41-c0989ab6615e None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:37.425667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f2856de3-d0ad-4d35-bc41-c0989ab6615e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:37.434060 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1217/2445] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 15:30:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:37.446529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5da760-8f36-4e79-97fe-5ad23116746f service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:37.459298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5da760-8f36-4e79-97fe-5ad23116746f service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:37.465883 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c157047-4b98-4672-81a2-a8606436e3ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:37.466517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c157047-4b98-4672-81a2-a8606436e3ce service swift] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:37.466798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c157047-4b98-4672-81a2-a8606436e3ce service swift] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:37.467071 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c157047-4b98-4672-81a2-a8606436e3ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:37.476454 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c157047-4b98-4672-81a2-a8606436e3ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:37.478450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c157047-4b98-4672-81a2-a8606436e3ce service swift] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:37.487593 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1229/2446] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:30:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:30:47.418745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=43 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:47.483946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=61 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:51.976462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e0ad4-44d2-43e4-8228-6acfbef72a9f None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:51.976931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e0ad4-44d2-43e4-8228-6acfbef72a9f None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:51.977024 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e0ad4-44d2-43e4-8228-6acfbef72a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:51.997588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2e2e0ad4-44d2-43e4-8228-6acfbef72a9f None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:51.998853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2e2e0ad4-44d2-43e4-8228-6acfbef72a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:52.007403 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1218/2447] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 15:30:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:52.020077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c157047-4b98-4672-81a2-a8606436e3ce service swift] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:30:52.035498 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c157047-4b98-4672-81a2-a8606436e3ce service swift] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:30:52.041661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d92f90-e4a0-42ec-ad5b-6d5f32db6ef3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:30:52.043204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d92f90-e4a0-42ec-ad5b-6d5f32db6ef3 service swift] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:52.043204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d92f90-e4a0-42ec-ad5b-6d5f32db6ef3 service swift] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:52.043204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d92f90-e4a0-42ec-ad5b-6d5f32db6ef3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:52.053608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d92f90-e4a0-42ec-ad5b-6d5f32db6ef3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:30:52.055511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d92f90-e4a0-42ec-ad5b-6d5f32db6ef3 service swift] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:30:52.065575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1230/2448] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:30:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:31:01.991647 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:02.062262 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=6 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:08.400487 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e0ad4-44d2-43e4-8228-6acfbef72a9f None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:08.402287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2e2e0ad4-44d2-43e4-8228-6acfbef72a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:08.414700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e0ad4-44d2-43e4-8228-6acfbef72a9f None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:08.418469 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebe154-dfcf-4b68-9f37-ae00b2129896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:08.419081 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebe154-dfcf-4b68-9f37-ae00b2129896 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:08.419356 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebe154-dfcf-4b68-9f37-ae00b2129896 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:08.419630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebe154-dfcf-4b68-9f37-ae00b2129896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:08.421678 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ebebe154-dfcf-4b68-9f37-ae00b2129896 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:08.491741 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebe154-dfcf-4b68-9f37-ae00b2129896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:08.493212 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebe154-dfcf-4b68-9f37-ae00b2129896 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:08.503484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1219/2449] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:09.394610 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d92f90-e4a0-42ec-ad5b-6d5f32db6ef3 service swift] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:09.405533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d92f90-e4a0-42ec-ad5b-6d5f32db6ef3 service swift] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:09.409560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32479eaa-d321-4c16-8fc3-9f25155b7a8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:09.410837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32479eaa-d321-4c16-8fc3-9f25155b7a8a service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:09.411274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32479eaa-d321-4c16-8fc3-9f25155b7a8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:09.411543 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32479eaa-d321-4c16-8fc3-9f25155b7a8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:09.413538 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-32479eaa-d321-4c16-8fc3-9f25155b7a8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:09.431002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32479eaa-d321-4c16-8fc3-9f25155b7a8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:09.432838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32479eaa-d321-4c16-8fc3-9f25155b7a8a service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:09.454202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1231/2450] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:12.367798 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebe154-dfcf-4b68-9f37-ae00b2129896 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:12.369346 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ebebe154-dfcf-4b68-9f37-ae00b2129896 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:12.508005 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32479eaa-d321-4c16-8fc3-9f25155b7a8a service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:12.511499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-32479eaa-d321-4c16-8fc3-9f25155b7a8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:12.524227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebe154-dfcf-4b68-9f37-ae00b2129896 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:12.543146 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32479eaa-d321-4c16-8fc3-9f25155b7a8a service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:12.543313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87faba-d23b-4260-ba2a-258c4327746f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '098d8d7e35804ccc931ddfba07f688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:12.544276 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87faba-d23b-4260-ba2a-258c4327746f service placement] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:12.544730 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87faba-d23b-4260-ba2a-258c4327746f service placement] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:12.549064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87faba-d23b-4260-ba2a-258c4327746f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:12.560817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-be87faba-d23b-4260-ba2a-258c4327746f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:12.580997 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b264bd-b55c-41cd-89e6-b8eec2af9705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:12.582833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87faba-d23b-4260-ba2a-258c4327746f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:12.584773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87faba-d23b-4260-ba2a-258c4327746f service placement] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:12.585846 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b264bd-b55c-41cd-89e6-b8eec2af9705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:12.590955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b264bd-b55c-41cd-89e6-b8eec2af9705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:12.591338 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b264bd-b55c-41cd-89e6-b8eec2af9705 admin admin] PATH_INFO: `/v3/users/d79f95d9699b4b0393e14a9212986ad5` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:12.593353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b264bd-b55c-41cd-89e6-b8eec2af9705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d79f95d9699b4b0393e14a9212986ad5)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:12.596363 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b264bd-b55c-41cd-89e6-b8eec2af9705 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:12.622816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1220/2451] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:12.697057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c2b264bd-b55c-41cd-89e6-b8eec2af9705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd79f95d9699b4b0393e14a9212986ad5'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:12.708145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c2b264bd-b55c-41cd-89e6-b8eec2af9705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd79f95d9699b4b0393e14a9212986ad5'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:12.709253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c2b264bd-b55c-41cd-89e6-b8eec2af9705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd79f95d9699b4b0393e14a9212986ad5'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:12.719703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c2b264bd-b55c-41cd-89e6-b8eec2af9705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd79f95d9699b4b0393e14a9212986ad5'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:12.736087 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c2b264bd-b55c-41cd-89e6-b8eec2af9705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd79f95d9699b4b0393e14a9212986ad5'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:12.746423 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1232/2452] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:31:12 2022] DELETE /identity/v3/users/d79f95d9699b4b0393e14a9212986ad5 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:31:13.000244 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87faba-d23b-4260-ba2a-258c4327746f service placement] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:13.026766 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87faba-d23b-4260-ba2a-258c4327746f service placement] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:13.106224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaf4b5-5767-48ec-9c83-49e2335a4264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:13.106789 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaf4b5-5767-48ec-9c83-49e2335a4264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:13.107230 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaf4b5-5767-48ec-9c83-49e2335a4264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:13.107511 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaf4b5-5767-48ec-9c83-49e2335a4264 admin admin] PATH_INFO: `/v3/projects/9d47732c77c34b63a94b3f682f420416` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:13.108809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfaf4b5-5767-48ec-9c83-49e2335a4264 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d47732c77c34b63a94b3f682f420416)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:13.110800 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfaf4b5-5767-48ec-9c83-49e2335a4264 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:13.157756 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:13.181218 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6bfaf4b5-5767-48ec-9c83-49e2335a4264 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d47732c77c34b63a94b3f682f420416', 'request_id': 'req-6bfaf4b5-5767-48ec-9c83-49e2335a4264'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:13.188954 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6bfaf4b5-5767-48ec-9c83-49e2335a4264 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d47732c77c34b63a94b3f682f420416', 'request_id': 'req-6bfaf4b5-5767-48ec-9c83-49e2335a4264'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:13.213138 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6bfaf4b5-5767-48ec-9c83-49e2335a4264 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d47732c77c34b63a94b3f682f420416', 'request_id': 'req-6bfaf4b5-5767-48ec-9c83-49e2335a4264'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:13.276869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6bfaf4b5-5767-48ec-9c83-49e2335a4264 admin admin] The token cache is being invalidate because project 9d47732c77c34b63a94b3f682f420416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:31:13.277308 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6bfaf4b5-5767-48ec-9c83-49e2335a4264 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:13.284568 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6bfaf4b5-5767-48ec-9c83-49e2335a4264 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:13.287781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1221/2453] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:31:12 2022] DELETE /identity/v3/projects/9d47732c77c34b63a94b3f682f420416 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:13.301571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cc5bc-b156-4251-92c0-ac3a4292a4b9 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:13.302921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cc5bc-b156-4251-92c0-ac3a4292a4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:13.303202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cc5bc-b156-4251-92c0-ac3a4292a4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:13.330252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-259cc5bc-b156-4251-92c0-ac3a4292a4b9 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:13.332592 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-259cc5bc-b156-4251-92c0-ac3a4292a4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:13.443913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1233/2454] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:13.452214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe462baf-68c1-4519-99db-ffa3bbf58045 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:13.452974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe462baf-68c1-4519-99db-ffa3bbf58045 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:13.452974 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe462baf-68c1-4519-99db-ffa3bbf58045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:13.481996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-fe462baf-68c1-4519-99db-ffa3bbf58045 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:13.483828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-fe462baf-68c1-4519-99db-ffa3bbf58045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:13.494360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1222/2455] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:31:13.497631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cc5bc-b156-4251-92c0-ac3a4292a4b9 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:13.511098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cc5bc-b156-4251-92c0-ac3a4292a4b9 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:13.528334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b52c44-8250-4a37-85fe-ed070a815408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:13.529744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b52c44-8250-4a37-85fe-ed070a815408 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:13.530357 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b52c44-8250-4a37-85fe-ed070a815408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:13.530844 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b52c44-8250-4a37-85fe-ed070a815408 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:13.533585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b52c44-8250-4a37-85fe-ed070a815408 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:31:13.534886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b52c44-8250-4a37-85fe-ed070a815408 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:13.537144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b52c44-8250-4a37-85fe-ed070a815408 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:13.557554 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1234/2456] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:31:13.563034 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe462baf-68c1-4519-99db-ffa3bbf58045 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:13.574542 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe462baf-68c1-4519-99db-ffa3bbf58045 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:13.581394 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665dc9aa-5cb2-4b83-8d6f-09377eeb7f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:13.582128 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dc9aa-5cb2-4b83-8d6f-09377eeb7f3e admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:13.583334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dc9aa-5cb2-4b83-8d6f-09377eeb7f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:13.583334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dc9aa-5cb2-4b83-8d6f-09377eeb7f3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:13.583753 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665dc9aa-5cb2-4b83-8d6f-09377eeb7f3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:13.585625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665dc9aa-5cb2-4b83-8d6f-09377eeb7f3e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:13.629144 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1223/2457] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:13 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:13.634857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b52c44-8250-4a37-85fe-ed070a815408 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:13.652520 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b52c44-8250-4a37-85fe-ed070a815408 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:13.716206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:13.739959 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfac5d-83c0-444c-96cf-8bfad1edb950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:13.740597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfac5d-83c0-444c-96cf-8bfad1edb950 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:13.741307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfac5d-83c0-444c-96cf-8bfad1edb950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:13.741307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dfac5d-83c0-444c-96cf-8bfad1edb950 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:13.741898 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dfac5d-83c0-444c-96cf-8bfad1edb950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:13.745618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dfac5d-83c0-444c-96cf-8bfad1edb950 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:13.796137 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1235/2458] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:13 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:31:13.798821 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665dc9aa-5cb2-4b83-8d6f-09377eeb7f3e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:13.828862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665dc9aa-5cb2-4b83-8d6f-09377eeb7f3e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:13.854654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fe5ba-500b-467e-98f5-da8c8506da6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:13.857043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fe5ba-500b-467e-98f5-da8c8506da6a admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:13.859206 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fe5ba-500b-467e-98f5-da8c8506da6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:13.860495 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fe5ba-500b-467e-98f5-da8c8506da6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:13.871130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5fe5ba-500b-467e-98f5-da8c8506da6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:13.874645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5fe5ba-500b-467e-98f5-da8c8506da6a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:13.889867 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1224/2459] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:13.894615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfac5d-83c0-444c-96cf-8bfad1edb950 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:13.940569 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dfac5d-83c0-444c-96cf-8bfad1edb950 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:13.947089 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb982d-2006-4301-93a5-d55dbbed748c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:13.947744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb982d-2006-4301-93a5-d55dbbed748c admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:13.948028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb982d-2006-4301-93a5-d55dbbed748c admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:13.948575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb982d-2006-4301-93a5-d55dbbed748c admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:13.949168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb982d-2006-4301-93a5-d55dbbed748c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:13.951060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb982d-2006-4301-93a5-d55dbbed748c admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:13.976367 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1236/2460] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:13.982945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fe5ba-500b-467e-98f5-da8c8506da6a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:13.999905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fe5ba-500b-467e-98f5-da8c8506da6a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:14.005092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ead70-8fe2-4841-9814-8b63b26a5758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:14.005940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ead70-8fe2-4841-9814-8b63b26a5758 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:14.006315 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ead70-8fe2-4841-9814-8b63b26a5758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:14.006720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ead70-8fe2-4841-9814-8b63b26a5758 admin admin] PATH_INFO: `/v3/projects/db8d0287e25f4873ba51a184f4748a41/users/cb2c553070cf423cb8e25657c8f5c648/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:14.023445 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ead70-8fe2-4841-9814-8b63b26a5758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db8d0287e25f4873ba51a184f4748a41, user_id=cb2c553070cf423cb8e25657c8f5c648, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:14.026755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ead70-8fe2-4841-9814-8b63b26a5758 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:14.042567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1225/2461] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:13 2022] PUT /identity/v3/projects/db8d0287e25f4873ba51a184f4748a41/users/cb2c553070cf423cb8e25657c8f5c648/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:14.052858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3bbf3-d98b-41dc-9648-53b524e3d96c None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:14.053181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3bbf3-d98b-41dc-9648-53b524e3d96c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:14.053415 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3bbf3-d98b-41dc-9648-53b524e3d96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:14.096740 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-aed3bbf3-d98b-41dc-9648-53b524e3d96c None None] MFA Rules not processed for user `cb2c553070cf423cb8e25657c8f5c648`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:14.098134 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-aed3bbf3-d98b-41dc-9648-53b524e3d96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:14.199856 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1237/2462] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:14.216430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ead70-8fe2-4841-9814-8b63b26a5758 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:14.219640 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bf4ead70-8fe2-4841-9814-8b63b26a5758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:14.321073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ead70-8fe2-4841-9814-8b63b26a5758 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:14.327678 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de19d5-9106-4608-b04d-138e32d04b58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:14.328318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de19d5-9106-4608-b04d-138e32d04b58 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:14.328594 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de19d5-9106-4608-b04d-138e32d04b58 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:14.328867 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de19d5-9106-4608-b04d-138e32d04b58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:14.339607 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de19d5-9106-4608-b04d-138e32d04b58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:14.341252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de19d5-9106-4608-b04d-138e32d04b58 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:14.352725 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1226/2463] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:14.422490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3bbf3-d98b-41dc-9648-53b524e3d96c None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:14.428467 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-aed3bbf3-d98b-41dc-9648-53b524e3d96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:14.533437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3bbf3-d98b-41dc-9648-53b524e3d96c None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:14.538468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab7356-cdbb-4923-aab4-47e037936471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:14.539084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab7356-cdbb-4923-aab4-47e037936471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:14.539355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab7356-cdbb-4923-aab4-47e037936471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:14.539635 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab7356-cdbb-4923-aab4-47e037936471 admin admin] PATH_INFO: `/v3/users/93d86ae4501e43d59d172d2e7acfdd54` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:14.540995 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab7356-cdbb-4923-aab4-47e037936471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93d86ae4501e43d59d172d2e7acfdd54)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:14.543271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab7356-cdbb-4923-aab4-47e037936471 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:14.615975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-daab7356-cdbb-4923-aab4-47e037936471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93d86ae4501e43d59d172d2e7acfdd54'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:14.624748 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-daab7356-cdbb-4923-aab4-47e037936471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93d86ae4501e43d59d172d2e7acfdd54'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:14.626195 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-daab7356-cdbb-4923-aab4-47e037936471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93d86ae4501e43d59d172d2e7acfdd54'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:14.628580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:14.640621 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-daab7356-cdbb-4923-aab4-47e037936471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93d86ae4501e43d59d172d2e7acfdd54'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:14.660355 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-daab7356-cdbb-4923-aab4-47e037936471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93d86ae4501e43d59d172d2e7acfdd54'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:14.670527 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1238/2464] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:31:14 2022] DELETE /identity/v3/users/93d86ae4501e43d59d172d2e7acfdd54 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:14.923462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de19d5-9106-4608-b04d-138e32d04b58 service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:14.941283 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de19d5-9106-4608-b04d-138e32d04b58 service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:14.987266 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:15.016092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3801c8-9994-459d-bdaa-818248436f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:15.016845 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3801c8-9994-459d-bdaa-818248436f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:15.017305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3801c8-9994-459d-bdaa-818248436f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:15.017686 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3801c8-9994-459d-bdaa-818248436f78 admin admin] PATH_INFO: `/v3/projects/3b8e7cb5fc4a4e598d27e51122265bda` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:15.019191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3801c8-9994-459d-bdaa-818248436f78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b8e7cb5fc4a4e598d27e51122265bda)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:15.021698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3801c8-9994-459d-bdaa-818248436f78 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:15.086969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9e3801c8-9994-459d-bdaa-818248436f78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b8e7cb5fc4a4e598d27e51122265bda', 'request_id': 'req-9e3801c8-9994-459d-bdaa-818248436f78'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:15.093065 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9e3801c8-9994-459d-bdaa-818248436f78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b8e7cb5fc4a4e598d27e51122265bda', 'request_id': 'req-9e3801c8-9994-459d-bdaa-818248436f78'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:15.101878 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9e3801c8-9994-459d-bdaa-818248436f78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b8e7cb5fc4a4e598d27e51122265bda', 'request_id': 'req-9e3801c8-9994-459d-bdaa-818248436f78'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:15.116997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9e3801c8-9994-459d-bdaa-818248436f78 admin admin] The token cache is being invalidate because project 3b8e7cb5fc4a4e598d27e51122265bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:31:15.117484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9e3801c8-9994-459d-bdaa-818248436f78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:15.118106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9e3801c8-9994-459d-bdaa-818248436f78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:15.119999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1227/2465] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:31:14 2022] DELETE /identity/v3/projects/3b8e7cb5fc4a4e598d27e51122265bda => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:15.127107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13f317-8590-40ec-84de-290c32c8ab0c None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:15.127560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13f317-8590-40ec-84de-290c32c8ab0c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:15.127777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13f317-8590-40ec-84de-290c32c8ab0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:15.150661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-2c13f317-8590-40ec-84de-290c32c8ab0c None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:15.151939 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2c13f317-8590-40ec-84de-290c32c8ab0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:15.231495 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1239/2466] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:15.246808 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212646cd-30dc-4256-bb2c-9ac275446f70 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:15.247084 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212646cd-30dc-4256-bb2c-9ac275446f70 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:15.247389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212646cd-30dc-4256-bb2c-9ac275446f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:15.267801 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-212646cd-30dc-4256-bb2c-9ac275446f70 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:15.269043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-212646cd-30dc-4256-bb2c-9ac275446f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:15.279421 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1228/2467] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:15.284002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13f317-8590-40ec-84de-290c32c8ab0c None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:15.294319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13f317-8590-40ec-84de-290c32c8ab0c None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:15.299725 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8101916-b4a7-4e7e-8af8-bf9700d1f814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:15.300358 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8101916-b4a7-4e7e-8af8-bf9700d1f814 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:15.300653 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8101916-b4a7-4e7e-8af8-bf9700d1f814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:15.300906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8101916-b4a7-4e7e-8af8-bf9700d1f814 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:15.301602 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8101916-b4a7-4e7e-8af8-bf9700d1f814 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:31:15.301971 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8101916-b4a7-4e7e-8af8-bf9700d1f814 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:15.303676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8101916-b4a7-4e7e-8af8-bf9700d1f814 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:15.316818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1240/2468] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:31:15.321929 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212646cd-30dc-4256-bb2c-9ac275446f70 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:15.333424 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212646cd-30dc-4256-bb2c-9ac275446f70 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:15.339968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822af17-f843-4f94-8c10-c79254ee6af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:15.340610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822af17-f843-4f94-8c10-c79254ee6af3 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:15.340892 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822af17-f843-4f94-8c10-c79254ee6af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:15.341163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822af17-f843-4f94-8c10-c79254ee6af3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:15.342154 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822af17-f843-4f94-8c10-c79254ee6af3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:15.344219 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822af17-f843-4f94-8c10-c79254ee6af3 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:15.374916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1229/2469] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:15 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:15.381574 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8101916-b4a7-4e7e-8af8-bf9700d1f814 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:15.392331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8101916-b4a7-4e7e-8af8-bf9700d1f814 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:15.457539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0abfe-4fa8-45ae-b11f-9475cc2e7954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:15.458242 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0abfe-4fa8-45ae-b11f-9475cc2e7954 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:15.458525 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0abfe-4fa8-45ae-b11f-9475cc2e7954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:15.458803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0abfe-4fa8-45ae-b11f-9475cc2e7954 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:15.459499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0abfe-4fa8-45ae-b11f-9475cc2e7954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:15.461610 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0abfe-4fa8-45ae-b11f-9475cc2e7954 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:15.492090 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1241/2470] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:15 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:31:15.496451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822af17-f843-4f94-8c10-c79254ee6af3 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:15.507865 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822af17-f843-4f94-8c10-c79254ee6af3 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:15.513544 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fdd5a-aa8b-4d00-b9dc-7d3ee14b20be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:15.514225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fdd5a-aa8b-4d00-b9dc-7d3ee14b20be admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:15.514511 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fdd5a-aa8b-4d00-b9dc-7d3ee14b20be admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:15.514806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fdd5a-aa8b-4d00-b9dc-7d3ee14b20be admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:15.515497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fdd5a-aa8b-4d00-b9dc-7d3ee14b20be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:15.517146 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fdd5a-aa8b-4d00-b9dc-7d3ee14b20be admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:15.529890 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1230/2471] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:15.534070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0abfe-4fa8-45ae-b11f-9475cc2e7954 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:15.544581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0abfe-4fa8-45ae-b11f-9475cc2e7954 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:15.549706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7953a3e-20d9-470d-8c52-9389dd641ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:15.550371 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7953a3e-20d9-470d-8c52-9389dd641ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:15.550653 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7953a3e-20d9-470d-8c52-9389dd641ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:15.550931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7953a3e-20d9-470d-8c52-9389dd641ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:15.551647 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7953a3e-20d9-470d-8c52-9389dd641ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:15.553253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7953a3e-20d9-470d-8c52-9389dd641ea8 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:15.566185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1242/2472] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:15.572139 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fdd5a-aa8b-4d00-b9dc-7d3ee14b20be admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:15.584861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fdd5a-aa8b-4d00-b9dc-7d3ee14b20be admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:15.592539 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e21c3-4c76-441f-b2b8-96dd9bcc1a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:15.593232 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e21c3-4c76-441f-b2b8-96dd9bcc1a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:15.593540 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e21c3-4c76-441f-b2b8-96dd9bcc1a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:15.593770 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e21c3-4c76-441f-b2b8-96dd9bcc1a56 admin admin] PATH_INFO: `/v3/projects/f1aea5fac11c44718d8fea0c6ede45c0/users/9a55b56f77674ac18e919d262f1bd4dd/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:15.612465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e21c3-4c76-441f-b2b8-96dd9bcc1a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1aea5fac11c44718d8fea0c6ede45c0, user_id=9a55b56f77674ac18e919d262f1bd4dd, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:15.614825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e21c3-4c76-441f-b2b8-96dd9bcc1a56 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:15.632746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1231/2473] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:15 2022] PUT /identity/v3/projects/f1aea5fac11c44718d8fea0c6ede45c0/users/9a55b56f77674ac18e919d262f1bd4dd/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:15.643266 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01e7e4-ac9e-482c-bf38-de30bf20ccc6 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:15.643830 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01e7e4-ac9e-482c-bf38-de30bf20ccc6 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:15.644277 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01e7e4-ac9e-482c-bf38-de30bf20ccc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:15.686437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-dc01e7e4-ac9e-482c-bf38-de30bf20ccc6 None None] MFA Rules not processed for user `9a55b56f77674ac18e919d262f1bd4dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:15.687683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-dc01e7e4-ac9e-482c-bf38-de30bf20ccc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:15.721948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:15.765732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1243/2474] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:15.777040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e21c3-4c76-441f-b2b8-96dd9bcc1a56 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:15.778979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d59e21c3-4c76-441f-b2b8-96dd9bcc1a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:15.852954 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:15.870997 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e21c3-4c76-441f-b2b8-96dd9bcc1a56 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:15.878094 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc65626-5ff9-4614-abca-871508d4c2b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:15.878830 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc65626-5ff9-4614-abca-871508d4c2b1 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:15.879108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc65626-5ff9-4614-abca-871508d4c2b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:15.879365 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc65626-5ff9-4614-abca-871508d4c2b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:15.888491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc65626-5ff9-4614-abca-871508d4c2b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:15.890149 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc65626-5ff9-4614-abca-871508d4c2b1 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:15.899855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1232/2475] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:16.561991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01e7e4-ac9e-482c-bf38-de30bf20ccc6 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:16.563632 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-dc01e7e4-ac9e-482c-bf38-de30bf20ccc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:16.637748 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01e7e4-ac9e-482c-bf38-de30bf20ccc6 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:16.641985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff0fde-f921-46f8-91a3-9e8ca0d8e0c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:16.642615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff0fde-f921-46f8-91a3-9e8ca0d8e0c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:16.642892 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff0fde-f921-46f8-91a3-9e8ca0d8e0c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:16.644033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff0fde-f921-46f8-91a3-9e8ca0d8e0c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:16.645468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d0ff0fde-f921-46f8-91a3-9e8ca0d8e0c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:16.710025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff0fde-f921-46f8-91a3-9e8ca0d8e0c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:16.711484 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff0fde-f921-46f8-91a3-9e8ca0d8e0c3 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:16.720641 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1244/2476] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:24.090432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc65626-5ff9-4614-abca-871508d4c2b1 service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:24.142142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff0fde-f921-46f8-91a3-9e8ca0d8e0c3 service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:24.142749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d0ff0fde-f921-46f8-91a3-9e8ca0d8e0c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:24.181317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc65626-5ff9-4614-abca-871508d4c2b1 service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:24.187442 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9a99f-3062-4794-86ed-815a0d4fce26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:24.188243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9a99f-3062-4794-86ed-815a0d4fce26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:24.188664 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9a99f-3062-4794-86ed-815a0d4fce26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:24.189134 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9a99f-3062-4794-86ed-815a0d4fce26 admin admin] PATH_INFO: `/v3/users/9a55b56f77674ac18e919d262f1bd4dd` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:24.190847 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9a99f-3062-4794-86ed-815a0d4fce26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a55b56f77674ac18e919d262f1bd4dd)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:24.193077 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9a99f-3062-4794-86ed-815a0d4fce26 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:24.259164 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff0fde-f921-46f8-91a3-9e8ca0d8e0c3 service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:24.264161 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-65f9a99f-3062-4794-86ed-815a0d4fce26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a55b56f77674ac18e919d262f1bd4dd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:24.265006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-65f9a99f-3062-4794-86ed-815a0d4fce26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a55b56f77674ac18e919d262f1bd4dd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:24.268483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c9c6f-5c77-45c0-8e44-1567cbf6883e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:24.270853 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c9c6f-5c77-45c0-8e44-1567cbf6883e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:24.271418 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c9c6f-5c77-45c0-8e44-1567cbf6883e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:24.271909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c9c6f-5c77-45c0-8e44-1567cbf6883e admin admin] PATH_INFO: `/v3/users/cb2c553070cf423cb8e25657c8f5c648` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:24.274317 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c9c6f-5c77-45c0-8e44-1567cbf6883e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb2c553070cf423cb8e25657c8f5c648)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:24.279783 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c9c6f-5c77-45c0-8e44-1567cbf6883e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:24.286662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-65f9a99f-3062-4794-86ed-815a0d4fce26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a55b56f77674ac18e919d262f1bd4dd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:24.297605 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-65f9a99f-3062-4794-86ed-815a0d4fce26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a55b56f77674ac18e919d262f1bd4dd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:24.307842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-65f9a99f-3062-4794-86ed-815a0d4fce26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a55b56f77674ac18e919d262f1bd4dd'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:24.316097 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1233/2477] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:31:24 2022] DELETE /identity/v3/users/9a55b56f77674ac18e919d262f1bd4dd => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:24.321176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8a409-fd74-478b-85e0-b73db462a9fc None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:24.321673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8a409-fd74-478b-85e0-b73db462a9fc None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:24.322123 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8a409-fd74-478b-85e0-b73db462a9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:24.331454 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:24.345271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-d6e8a409-fd74-478b-85e0-b73db462a9fc None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:24.346553 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d6e8a409-fd74-478b-85e0-b73db462a9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:24.362682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-638c9c6f-5c77-45c0-8e44-1567cbf6883e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb2c553070cf423cb8e25657c8f5c648'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:24.372833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-638c9c6f-5c77-45c0-8e44-1567cbf6883e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb2c553070cf423cb8e25657c8f5c648'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:24.373551 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-638c9c6f-5c77-45c0-8e44-1567cbf6883e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb2c553070cf423cb8e25657c8f5c648'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:24.384083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-638c9c6f-5c77-45c0-8e44-1567cbf6883e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb2c553070cf423cb8e25657c8f5c648'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:24.406356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-638c9c6f-5c77-45c0-8e44-1567cbf6883e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb2c553070cf423cb8e25657c8f5c648'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:24.416364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1245/2478] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:31:24 2022] DELETE /identity/v3/users/cb2c553070cf423cb8e25657c8f5c648 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:24.423565 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60982d4f-5049-4282-bad7-805acd0df850 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:24.424060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60982d4f-5049-4282-bad7-805acd0df850 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:24.424421 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60982d4f-5049-4282-bad7-805acd0df850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:24.430635 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1234/2479] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:31:24.444690 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8a409-fd74-478b-85e0-b73db462a9fc None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:24.447265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d6e8a409-fd74-478b-85e0-b73db462a9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:24.455854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-60982d4f-5049-4282-bad7-805acd0df850 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:24.457128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-60982d4f-5049-4282-bad7-805acd0df850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:24.461175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:24.541650 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8a409-fd74-478b-85e0-b73db462a9fc None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:24.548422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1246/2480] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:24.564153 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60982d4f-5049-4282-bad7-805acd0df850 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:24.640147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c513b0-7b72-455e-a5d8-ca97057853d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:24.640768 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c513b0-7b72-455e-a5d8-ca97057853d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:24.641017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c513b0-7b72-455e-a5d8-ca97057853d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:24.641286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c513b0-7b72-455e-a5d8-ca97057853d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:24.656937 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c513b0-7b72-455e-a5d8-ca97057853d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:24.658007 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60982d4f-5049-4282-bad7-805acd0df850 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:24.659634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c513b0-7b72-455e-a5d8-ca97057853d2 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:24.662580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ace6d8-f279-49b2-82ef-627eb46ea21a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:24.663444 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ace6d8-f279-49b2-82ef-627eb46ea21a service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:24.663732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ace6d8-f279-49b2-82ef-627eb46ea21a service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:24.664137 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ace6d8-f279-49b2-82ef-627eb46ea21a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:24.674381 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1235/2481] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:31:24.676428 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ace6d8-f279-49b2-82ef-627eb46ea21a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:24.678630 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ace6d8-f279-49b2-82ef-627eb46ea21a service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:24.700328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1247/2482] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:25.100918 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c513b0-7b72-455e-a5d8-ca97057853d2 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:25.102737 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d2c513b0-7b72-455e-a5d8-ca97057853d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:25.125895 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c513b0-7b72-455e-a5d8-ca97057853d2 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:25.134095 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608e300-4e9b-40bb-b228-e06468f79c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:25.136199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608e300-4e9b-40bb-b228-e06468f79c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:25.136732 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608e300-4e9b-40bb-b228-e06468f79c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:25.137416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608e300-4e9b-40bb-b228-e06468f79c3b admin admin] PATH_INFO: `/v3/projects/f1aea5fac11c44718d8fea0c6ede45c0` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:25.140362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608e300-4e9b-40bb-b228-e06468f79c3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1aea5fac11c44718d8fea0c6ede45c0)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:25.144677 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608e300-4e9b-40bb-b228-e06468f79c3b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:25.203877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ace6d8-f279-49b2-82ef-627eb46ea21a service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:25.206621 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c8ace6d8-f279-49b2-82ef-627eb46ea21a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:25.229181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ace6d8-f279-49b2-82ef-627eb46ea21a service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:25.237218 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6608e300-4e9b-40bb-b228-e06468f79c3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1aea5fac11c44718d8fea0c6ede45c0', 'request_id': 'req-6608e300-4e9b-40bb-b228-e06468f79c3b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:25.252015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6608e300-4e9b-40bb-b228-e06468f79c3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1aea5fac11c44718d8fea0c6ede45c0', 'request_id': 'req-6608e300-4e9b-40bb-b228-e06468f79c3b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:25.263564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6608e300-4e9b-40bb-b228-e06468f79c3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1aea5fac11c44718d8fea0c6ede45c0', 'request_id': 'req-6608e300-4e9b-40bb-b228-e06468f79c3b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:25.280296 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6608e300-4e9b-40bb-b228-e06468f79c3b admin admin] The token cache is being invalidate because project f1aea5fac11c44718d8fea0c6ede45c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:31:25.280615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6608e300-4e9b-40bb-b228-e06468f79c3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:25.281264 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-6608e300-4e9b-40bb-b228-e06468f79c3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:25.283612 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1236/2483] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:31:25 2022] DELETE /identity/v3/projects/f1aea5fac11c44718d8fea0c6ede45c0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:25.289118 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9d7a2-004e-4fb5-8e2a-625a6d7653a3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:25.289455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9d7a2-004e-4fb5-8e2a-625a6d7653a3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:25.289672 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9d7a2-004e-4fb5-8e2a-625a6d7653a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:25.297836 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:25.314077 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-00a9d7a2-004e-4fb5-8e2a-625a6d7653a3 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:25.317012 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-00a9d7a2-004e-4fb5-8e2a-625a6d7653a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:25.317287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b8c0d-22b1-4bea-8a09-d229c5e9eb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:25.318018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b8c0d-22b1-4bea-8a09-d229c5e9eb75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:25.318336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b8c0d-22b1-4bea-8a09-d229c5e9eb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:25.318566 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b8c0d-22b1-4bea-8a09-d229c5e9eb75 admin admin] PATH_INFO: `/v3/projects/db8d0287e25f4873ba51a184f4748a41` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:25.320121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b8c0d-22b1-4bea-8a09-d229c5e9eb75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db8d0287e25f4873ba51a184f4748a41)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:25.322267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b8c0d-22b1-4bea-8a09-d229c5e9eb75 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:25.326875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1237/2484] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:31:25.338574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4d9a6-0eec-4eef-979d-5a80d7da6a74 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:25.338851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4d9a6-0eec-4eef-979d-5a80d7da6a74 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:25.339126 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4d9a6-0eec-4eef-979d-5a80d7da6a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:25.372208 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5fd4d9a6-0eec-4eef-979d-5a80d7da6a74 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:25.374270 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5fd4d9a6-0eec-4eef-979d-5a80d7da6a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:25.395127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1238/2485] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:31:25.400205 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4d9a6-0eec-4eef-979d-5a80d7da6a74 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:25.403795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8d5b8c0d-22b1-4bea-8a09-d229c5e9eb75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db8d0287e25f4873ba51a184f4748a41', 'request_id': 'req-8d5b8c0d-22b1-4bea-8a09-d229c5e9eb75'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:25.412906 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4d9a6-0eec-4eef-979d-5a80d7da6a74 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:25.416105 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8d5b8c0d-22b1-4bea-8a09-d229c5e9eb75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db8d0287e25f4873ba51a184f4748a41', 'request_id': 'req-8d5b8c0d-22b1-4bea-8a09-d229c5e9eb75'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:25.426290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8d5b8c0d-22b1-4bea-8a09-d229c5e9eb75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db8d0287e25f4873ba51a184f4748a41', 'request_id': 'req-8d5b8c0d-22b1-4bea-8a09-d229c5e9eb75'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:25.444505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8d5b8c0d-22b1-4bea-8a09-d229c5e9eb75 admin admin] The token cache is being invalidate because project db8d0287e25f4873ba51a184f4748a41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:31:25.444828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8d5b8c0d-22b1-4bea-8a09-d229c5e9eb75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:25.445369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8d5b8c0d-22b1-4bea-8a09-d229c5e9eb75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:25.447928 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1248/2486] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:31:25 2022] DELETE /identity/v3/projects/db8d0287e25f4873ba51a184f4748a41 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:25.455148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571a421-382a-49d6-8f6c-d68469e01b0f None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:25.455414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571a421-382a-49d6-8f6c-d68469e01b0f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:25.455684 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571a421-382a-49d6-8f6c-d68469e01b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:25.464925 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:25.480884 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-e571a421-382a-49d6-8f6c-d68469e01b0f None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:25.482762 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e571a421-382a-49d6-8f6c-d68469e01b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:25.503848 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a3833-ae72-4c67-ba12-1502fc0514f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:25.504675 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a3833-ae72-4c67-ba12-1502fc0514f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:25.505018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a3833-ae72-4c67-ba12-1502fc0514f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:25.505357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a3833-ae72-4c67-ba12-1502fc0514f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:25.506220 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a3833-ae72-4c67-ba12-1502fc0514f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:31:25.506743 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a3833-ae72-4c67-ba12-1502fc0514f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:25.508553 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a3833-ae72-4c67-ba12-1502fc0514f5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:25.523306 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1239/2487] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:31:25.529280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a3833-ae72-4c67-ba12-1502fc0514f5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:25.531186 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-462a3833-ae72-4c67-ba12-1502fc0514f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:25.547531 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a3833-ae72-4c67-ba12-1502fc0514f5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:25.553265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1315431-ea5e-47d5-9f77-775cd81e6465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:25.554076 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1315431-ea5e-47d5-9f77-775cd81e6465 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:25.554358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1315431-ea5e-47d5-9f77-775cd81e6465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:25.554721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1315431-ea5e-47d5-9f77-775cd81e6465 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:25.555413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1315431-ea5e-47d5-9f77-775cd81e6465 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:25.558245 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1315431-ea5e-47d5-9f77-775cd81e6465 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:25.565075 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1249/2488] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:31:25.575246 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd7fe5-51a6-4db9-83fd-3a04dde64993 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:25.575527 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd7fe5-51a6-4db9-83fd-3a04dde64993 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:25.575794 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd7fe5-51a6-4db9-83fd-3a04dde64993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:25.600280 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1240/2489] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:25 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:31:25.606282 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1315431-ea5e-47d5-9f77-775cd81e6465 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:25.606927 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-71bd7fe5-51a6-4db9-83fd-3a04dde64993 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:25.608308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-71bd7fe5-51a6-4db9-83fd-3a04dde64993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:25.727941 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1315431-ea5e-47d5-9f77-775cd81e6465 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:25.729951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1250/2490] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:31:25.733263 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd7fe5-51a6-4db9-83fd-3a04dde64993 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:25.736278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c61b2-751a-43fe-a87c-0b885bd27b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:25.737252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c61b2-751a-43fe-a87c-0b885bd27b06 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:25.737802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c61b2-751a-43fe-a87c-0b885bd27b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:25.738300 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c61b2-751a-43fe-a87c-0b885bd27b06 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:25.739370 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c61b2-751a-43fe-a87c-0b885bd27b06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:25.742844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c61b2-751a-43fe-a87c-0b885bd27b06 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:25.755981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd7fe5-51a6-4db9-83fd-3a04dde64993 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:25.773543 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1acafc-fcb8-454d-ab02-cf335d02e1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:25.774544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1acafc-fcb8-454d-ab02-cf335d02e1da admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:25.774860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1acafc-fcb8-454d-ab02-cf335d02e1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:25.775134 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1acafc-fcb8-454d-ab02-cf335d02e1da admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:25.776466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1acafc-fcb8-454d-ab02-cf335d02e1da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:31:25.776935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1acafc-fcb8-454d-ab02-cf335d02e1da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:25.779267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1acafc-fcb8-454d-ab02-cf335d02e1da admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:25.782599 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1241/2491] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:25 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:31:25.792371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c61b2-751a-43fe-a87c-0b885bd27b06 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:25.806559 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1251/2492] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:31:25.809380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1acafc-fcb8-454d-ab02-cf335d02e1da admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:25.821030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:25.822801 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c61b2-751a-43fe-a87c-0b885bd27b06 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:25.826165 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1acafc-fcb8-454d-ab02-cf335d02e1da admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:25.831758 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd1941-b462-42d8-915e-1fecc5d0d49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:25.832465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd1941-b462-42d8-915e-1fecc5d0d49e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:25.832722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd1941-b462-42d8-915e-1fecc5d0d49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:25.832972 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd1941-b462-42d8-915e-1fecc5d0d49e admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:25.833686 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd1941-b462-42d8-915e-1fecc5d0d49e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:25.834202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b32e5-425e-44d7-a303-1c80d2dd44b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:25.834915 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b32e5-425e-44d7-a303-1c80d2dd44b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:25.835272 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b32e5-425e-44d7-a303-1c80d2dd44b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:25.836529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b32e5-425e-44d7-a303-1c80d2dd44b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:25.837399 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd1941-b462-42d8-915e-1fecc5d0d49e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:25.841596 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b32e5-425e-44d7-a303-1c80d2dd44b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:25.841596 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b32e5-425e-44d7-a303-1c80d2dd44b8 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:25.853750 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1242/2493] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:25.859562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd1941-b462-42d8-915e-1fecc5d0d49e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:25.875442 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd1941-b462-42d8-915e-1fecc5d0d49e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:25.883564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de041f66-2070-48e4-8650-f7c145aa897b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:25.884398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de041f66-2070-48e4-8650-f7c145aa897b admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:25.884820 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de041f66-2070-48e4-8650-f7c145aa897b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:25.890063 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1252/2494] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:25 2022] POST /identity/v3/projects => generated 381 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:25.890318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de041f66-2070-48e4-8650-f7c145aa897b admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:25.890318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de041f66-2070-48e4-8650-f7c145aa897b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:25.890318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de041f66-2070-48e4-8650-f7c145aa897b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:25.891177 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b32e5-425e-44d7-a303-1c80d2dd44b8 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:25.907842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1243/2495] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:25.912856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de041f66-2070-48e4-8650-f7c145aa897b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:25.927442 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de041f66-2070-48e4-8650-f7c145aa897b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:25.984406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b32e5-425e-44d7-a303-1c80d2dd44b8 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:25.993308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d7e40-04fe-4fc3-a644-d99560bb9914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:25.993878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d7e40-04fe-4fc3-a644-d99560bb9914 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:25.994361 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d7e40-04fe-4fc3-a644-d99560bb9914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:25.994876 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d7e40-04fe-4fc3-a644-d99560bb9914 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:25.995748 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d7e40-04fe-4fc3-a644-d99560bb9914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:25.998179 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d7e40-04fe-4fc3-a644-d99560bb9914 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:26.017215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cedcad0-6269-4cc2-8837-8da806fa9f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:26.018176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedcad0-6269-4cc2-8837-8da806fa9f22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:26.018449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedcad0-6269-4cc2-8837-8da806fa9f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:26.018759 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedcad0-6269-4cc2-8837-8da806fa9f22 admin admin] PATH_INFO: `/v3/projects/12332bafe0274f9f9afd715805c03742/users/7fc69bc54b4f43bd9ba09cc9b46e11ca/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:26.030682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1253/2496] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:25 2022] POST /identity/v3/users => generated 291 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:26.038014 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d7e40-04fe-4fc3-a644-d99560bb9914 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:26.051678 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cedcad0-6269-4cc2-8837-8da806fa9f22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12332bafe0274f9f9afd715805c03742, user_id=7fc69bc54b4f43bd9ba09cc9b46e11ca, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:26.053921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d7e40-04fe-4fc3-a644-d99560bb9914 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:26.057159 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cedcad0-6269-4cc2-8837-8da806fa9f22 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:26.061064 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde650d-0e52-4067-a605-753b3bffdbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:26.061987 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde650d-0e52-4067-a605-753b3bffdbc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:26.062338 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde650d-0e52-4067-a605-753b3bffdbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:26.062744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde650d-0e52-4067-a605-753b3bffdbc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:26.065333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde650d-0e52-4067-a605-753b3bffdbc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:26.068409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde650d-0e52-4067-a605-753b3bffdbc2 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:26.076270 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:26.084363 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1244/2497] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:25 2022] PUT /identity/v3/projects/12332bafe0274f9f9afd715805c03742/users/7fc69bc54b4f43bd9ba09cc9b46e11ca/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:31:26.093965 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1254/2498] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:31:26.096048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cedcad0-6269-4cc2-8837-8da806fa9f22 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:26.096777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0823253-5ee8-40fc-b39b-427ebae41beb None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:26.097322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0823253-5ee8-40fc-b39b-427ebae41beb None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:26.097739 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0823253-5ee8-40fc-b39b-427ebae41beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:26.139852 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-a0823253-5ee8-40fc-b39b-427ebae41beb None None] MFA Rules not processed for user `7fc69bc54b4f43bd9ba09cc9b46e11ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:26.143045 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-a0823253-5ee8-40fc-b39b-427ebae41beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:26.187982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cedcad0-6269-4cc2-8837-8da806fa9f22 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:26.194930 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd46b9-5962-42bf-abc8-c1cb95db2131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:26.195622 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd46b9-5962-42bf-abc8-c1cb95db2131 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:26.195906 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd46b9-5962-42bf-abc8-c1cb95db2131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:26.196181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd46b9-5962-42bf-abc8-c1cb95db2131 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:26.196913 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd46b9-5962-42bf-abc8-c1cb95db2131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:26.198928 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd46b9-5962-42bf-abc8-c1cb95db2131 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:26.213571 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1245/2499] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:31:26.218001 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd46b9-5962-42bf-abc8-c1cb95db2131 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:26.237903 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd46b9-5962-42bf-abc8-c1cb95db2131 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:26.244389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d17a6-5ce8-4124-9efd-1509b6f2b021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:26.245491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d17a6-5ce8-4124-9efd-1509b6f2b021 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:26.245989 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d17a6-5ce8-4124-9efd-1509b6f2b021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:26.246441 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d17a6-5ce8-4124-9efd-1509b6f2b021 admin admin] PATH_INFO: `/v3/projects/4896130f6b08478ba5c6dff0379414c0/users/7d1ce5135c64437cb48b1e1c14cfec6c/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:26.268159 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d17a6-5ce8-4124-9efd-1509b6f2b021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4896130f6b08478ba5c6dff0379414c0, user_id=7d1ce5135c64437cb48b1e1c14cfec6c, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:26.269482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1255/2500] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:31:26.271151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d17a6-5ce8-4124-9efd-1509b6f2b021 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:26.277056 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0823253-5ee8-40fc-b39b-427ebae41beb None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:26.291381 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0823253-5ee8-40fc-b39b-427ebae41beb None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:26.296994 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1246/2501] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:26 2022] PUT /identity/v3/projects/4896130f6b08478ba5c6dff0379414c0/users/7d1ce5135c64437cb48b1e1c14cfec6c/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:31:26.308468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba040e24-d5bd-4c16-91e5-9cb0163cf310 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:26.308780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba040e24-d5bd-4c16-91e5-9cb0163cf310 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:26.309023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba040e24-d5bd-4c16-91e5-9cb0163cf310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:26.344377 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-ba040e24-d5bd-4c16-91e5-9cb0163cf310 None None] MFA Rules not processed for user `7d1ce5135c64437cb48b1e1c14cfec6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:26.346171 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ba040e24-d5bd-4c16-91e5-9cb0163cf310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:26.352476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:26.367737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e66114-d29e-4a2d-ac71-ec6cb442c387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:26.368384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e66114-d29e-4a2d-ac71-ec6cb442c387 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:26.368696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e66114-d29e-4a2d-ac71-ec6cb442c387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:26.368970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e66114-d29e-4a2d-ac71-ec6cb442c387 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:26.369669 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e66114-d29e-4a2d-ac71-ec6cb442c387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:26.371768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e66114-d29e-4a2d-ac71-ec6cb442c387 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:26.413955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1256/2502] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:26 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:26.415363 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e66114-d29e-4a2d-ac71-ec6cb442c387 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:26.456829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1247/2503] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:31:26.464563 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba040e24-d5bd-4c16-91e5-9cb0163cf310 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:26.487821 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba040e24-d5bd-4c16-91e5-9cb0163cf310 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:26.532181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e66114-d29e-4a2d-ac71-ec6cb442c387 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:26.543190 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c2113-2811-45c9-8244-9610767f2fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:26.544037 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c2113-2811-45c9-8244-9610767f2fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:26.544470 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c2113-2811-45c9-8244-9610767f2fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:26.545265 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c2113-2811-45c9-8244-9610767f2fc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:26.546352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c2113-2811-45c9-8244-9610767f2fc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:26.550223 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c2113-2811-45c9-8244-9610767f2fc7 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:26.582909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1257/2504] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:26 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:26.587768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c2113-2811-45c9-8244-9610767f2fc7 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:26.594475 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a9c83-dbca-4cc1-8cd0-1e4e3a5c5ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:26.595380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a9c83-dbca-4cc1-8cd0-1e4e3a5c5ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:26.595662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a9c83-dbca-4cc1-8cd0-1e4e3a5c5ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:26.595926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a9c83-dbca-4cc1-8cd0-1e4e3a5c5ed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:26.596645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a9c83-dbca-4cc1-8cd0-1e4e3a5c5ed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:26.599634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c2113-2811-45c9-8244-9610767f2fc7 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:26.600168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a9c83-dbca-4cc1-8cd0-1e4e3a5c5ed7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:26.607961 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:26.611066 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1754eb-e17e-4ca6-8224-afe935a31a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:26.612393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1754eb-e17e-4ca6-8224-afe935a31a27 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:26.612900 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1754eb-e17e-4ca6-8224-afe935a31a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:26.613378 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1754eb-e17e-4ca6-8224-afe935a31a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:26.614763 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1754eb-e17e-4ca6-8224-afe935a31a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:26.618037 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1754eb-e17e-4ca6-8224-afe935a31a27 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:26.635451 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1258/2505] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:31:26.643301 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1754eb-e17e-4ca6-8224-afe935a31a27 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:26.646327 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1248/2506] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:26 2022] POST /identity/v3/projects => generated 381 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:31:26.651899 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a9c83-dbca-4cc1-8cd0-1e4e3a5c5ed7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:26.655947 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1754eb-e17e-4ca6-8224-afe935a31a27 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:26.766183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a9c83-dbca-4cc1-8cd0-1e4e3a5c5ed7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:26.772141 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83a15a-3098-4f90-8877-c37e0f12398c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:26.772967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83a15a-3098-4f90-8877-c37e0f12398c admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:26.773160 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83a15a-3098-4f90-8877-c37e0f12398c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:26.773628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d83a15a-3098-4f90-8877-c37e0f12398c admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:26.774716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83a15a-3098-4f90-8877-c37e0f12398c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:26.777094 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fdce5-7918-4ffd-a21f-96e5ae9459f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:26.777930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fdce5-7918-4ffd-a21f-96e5ae9459f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:26.778126 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d83a15a-3098-4f90-8877-c37e0f12398c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:26.778287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fdce5-7918-4ffd-a21f-96e5ae9459f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:26.778469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4fdce5-7918-4ffd-a21f-96e5ae9459f2 admin admin] PATH_INFO: `/v3/projects/e136bf4f0a2445a69a1ffeb09198858d/users/d6b08f5f9a6e48be8202afdfcfe502f4/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:26.797104 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4fdce5-7918-4ffd-a21f-96e5ae9459f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e136bf4f0a2445a69a1ffeb09198858d, user_id=d6b08f5f9a6e48be8202afdfcfe502f4, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:26.800438 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4fdce5-7918-4ffd-a21f-96e5ae9459f2 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:26.807910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1249/2507] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:26 2022] POST /identity/v3/users => generated 297 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:26.812249 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83a15a-3098-4f90-8877-c37e0f12398c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:26.825367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d83a15a-3098-4f90-8877-c37e0f12398c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:26.827152 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1259/2508] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:26 2022] PUT /identity/v3/projects/e136bf4f0a2445a69a1ffeb09198858d/users/d6b08f5f9a6e48be8202afdfcfe502f4/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:31:26.834230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fdce5-7918-4ffd-a21f-96e5ae9459f2 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:26.900860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:26.913106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fced40-6e9d-416d-98e9-63edca1e205e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:26.914178 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fced40-6e9d-416d-98e9-63edca1e205e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:26.914667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fced40-6e9d-416d-98e9-63edca1e205e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:26.915125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fced40-6e9d-416d-98e9-63edca1e205e admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:26.916363 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fced40-6e9d-416d-98e9-63edca1e205e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:26.919195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fced40-6e9d-416d-98e9-63edca1e205e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:26.935198 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4fdce5-7918-4ffd-a21f-96e5ae9459f2 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:26.935457 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1250/2509] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:31:26.936854 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23100bf3-dee4-4e5f-8f95-7a37a26ea468 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:26.937079 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23100bf3-dee4-4e5f-8f95-7a37a26ea468 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:26.937324 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23100bf3-dee4-4e5f-8f95-7a37a26ea468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:26.944789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe53e12-faac-4567-b283-364dc31accb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:26.946203 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe53e12-faac-4567-b283-364dc31accb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:26.946630 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe53e12-faac-4567-b283-364dc31accb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:26.947137 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe53e12-faac-4567-b283-364dc31accb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:26.948214 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe53e12-faac-4567-b283-364dc31accb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:26.950561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe53e12-faac-4567-b283-364dc31accb7 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:26.957944 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-23100bf3-dee4-4e5f-8f95-7a37a26ea468 None None] MFA Rules not processed for user `0898fda8be3743f093dbb216f21eb931`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:26.959194 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-23100bf3-dee4-4e5f-8f95-7a37a26ea468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:26.965755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1260/2510] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:26.966798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe53e12-faac-4567-b283-364dc31accb7 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:26.978756 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe53e12-faac-4567-b283-364dc31accb7 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:26.983963 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae115d-89a4-4945-ac33-87716a26adc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:26.984680 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae115d-89a4-4945-ac33-87716a26adc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:26.984939 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae115d-89a4-4945-ac33-87716a26adc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:26.985222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae115d-89a4-4945-ac33-87716a26adc7 admin admin] PATH_INFO: `/v3/projects/ef3dfc6bd9b7439d9b39ceae75b5f588/users/fe9920820c314477beeda6e9aea4da2e/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:27.004001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae115d-89a4-4945-ac33-87716a26adc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef3dfc6bd9b7439d9b39ceae75b5f588, user_id=fe9920820c314477beeda6e9aea4da2e, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:27.007561 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae115d-89a4-4945-ac33-87716a26adc7 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:27.023752 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1261/2511] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:26 2022] PUT /identity/v3/projects/ef3dfc6bd9b7439d9b39ceae75b5f588/users/fe9920820c314477beeda6e9aea4da2e/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:27.024252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae115d-89a4-4945-ac33-87716a26adc7 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:27.047270 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1251/2512] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:27.047786 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23100bf3-dee4-4e5f-8f95-7a37a26ea468 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:27.067674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23100bf3-dee4-4e5f-8f95-7a37a26ea468 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:27.086440 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:27.117307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae115d-89a4-4945-ac33-87716a26adc7 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:27.123366 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270804cc-0aa6-44f3-8b8c-1beb183084eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:27.124346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270804cc-0aa6-44f3-8b8c-1beb183084eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:27.124468 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270804cc-0aa6-44f3-8b8c-1beb183084eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:27.125530 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270804cc-0aa6-44f3-8b8c-1beb183084eb admin admin] PATH_INFO: `/v3/projects/e136bf4f0a2445a69a1ffeb09198858d/users/d6b08f5f9a6e48be8202afdfcfe502f4/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:27.127777 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270804cc-0aa6-44f3-8b8c-1beb183084eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e136bf4f0a2445a69a1ffeb09198858d, user_id=d6b08f5f9a6e48be8202afdfcfe502f4, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:27.130822 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270804cc-0aa6-44f3-8b8c-1beb183084eb admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:27.145598 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1262/2513] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:27 2022] PUT /identity/v3/projects/e136bf4f0a2445a69a1ffeb09198858d/users/d6b08f5f9a6e48be8202afdfcfe502f4/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:27.163465 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d31257-5526-4cfe-b11d-ea6f06f8bb51 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:27.163907 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d31257-5526-4cfe-b11d-ea6f06f8bb51 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:27.164268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d31257-5526-4cfe-b11d-ea6f06f8bb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:27.183863 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3699ee-0332-440b-9fe7-e15a835a27b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:27.184836 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3699ee-0332-440b-9fe7-e15a835a27b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:27.185271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3699ee-0332-440b-9fe7-e15a835a27b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:27.185686 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3699ee-0332-440b-9fe7-e15a835a27b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:27.186593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3699ee-0332-440b-9fe7-e15a835a27b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:27.188543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3699ee-0332-440b-9fe7-e15a835a27b8 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:27.208077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-06d31257-5526-4cfe-b11d-ea6f06f8bb51 None None] MFA Rules not processed for user `d6b08f5f9a6e48be8202afdfcfe502f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:27.210206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-06d31257-5526-4cfe-b11d-ea6f06f8bb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:27.213925 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1252/2514] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:27.215665 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3699ee-0332-440b-9fe7-e15a835a27b8 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:27.234247 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3699ee-0332-440b-9fe7-e15a835a27b8 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:27.320134 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1263/2515] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:27.331647 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f11b21-2179-4aeb-aad9-16b249032e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:27.332660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f11b21-2179-4aeb-aad9-16b249032e07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:27.332660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f11b21-2179-4aeb-aad9-16b249032e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:27.333072 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f11b21-2179-4aeb-aad9-16b249032e07 admin admin] PATH_INFO: `/v3/projects/ef3dfc6bd9b7439d9b39ceae75b5f588/users/fe9920820c314477beeda6e9aea4da2e/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:27.335549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f11b21-2179-4aeb-aad9-16b249032e07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef3dfc6bd9b7439d9b39ceae75b5f588, user_id=fe9920820c314477beeda6e9aea4da2e, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:27.337487 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d31257-5526-4cfe-b11d-ea6f06f8bb51 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:27.338820 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f11b21-2179-4aeb-aad9-16b249032e07 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:27.339176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-06d31257-5526-4cfe-b11d-ea6f06f8bb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:27.354900 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1253/2516] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:27 2022] PUT /identity/v3/projects/ef3dfc6bd9b7439d9b39ceae75b5f588/users/fe9920820c314477beeda6e9aea4da2e/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:27.372282 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be1e12-1807-4f0c-884a-f8fff8d485b0 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:27.372654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be1e12-1807-4f0c-884a-f8fff8d485b0 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:27.373015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be1e12-1807-4f0c-884a-f8fff8d485b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:27.403354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:27.423696 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f1be1e12-1807-4f0c-884a-f8fff8d485b0 None None] MFA Rules not processed for user `fe9920820c314477beeda6e9aea4da2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:27.426064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f1be1e12-1807-4f0c-884a-f8fff8d485b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:27.462216 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d31257-5526-4cfe-b11d-ea6f06f8bb51 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:27.540756 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be34446-4e5d-44b9-bd26-b54bd79b652a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:27.541280 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be34446-4e5d-44b9-bd26-b54bd79b652a service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:27.541941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be34446-4e5d-44b9-bd26-b54bd79b652a service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:27.542297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be34446-4e5d-44b9-bd26-b54bd79b652a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:27.556641 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1254/2517] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:27.571271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be1e12-1807-4f0c-884a-f8fff8d485b0 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:27.572831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f1be1e12-1807-4f0c-884a-f8fff8d485b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:27.587148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be1e12-1807-4f0c-884a-f8fff8d485b0 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:27.592313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a1db9-824e-4aef-869f-07d7776f6121 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:27.593262 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a1db9-824e-4aef-869f-07d7776f6121 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:27.593818 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a1db9-824e-4aef-869f-07d7776f6121 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:27.594236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a1db9-824e-4aef-869f-07d7776f6121 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:27.613043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a1db9-824e-4aef-869f-07d7776f6121 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:27.615982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a1db9-824e-4aef-869f-07d7776f6121 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:27.640509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be34446-4e5d-44b9-bd26-b54bd79b652a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:27.641303 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1255/2518] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:31:27.643415 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be34446-4e5d-44b9-bd26-b54bd79b652a service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:27.662107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1264/2519] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:31:27.758201 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a1db9-824e-4aef-869f-07d7776f6121 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:27.771232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be34446-4e5d-44b9-bd26-b54bd79b652a service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:27.776608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a1db9-824e-4aef-869f-07d7776f6121 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:27.780771 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6554a11-d401-460a-95e5-18ae1a5b557c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:27.781410 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6554a11-d401-460a-95e5-18ae1a5b557c service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:27.781685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6554a11-d401-460a-95e5-18ae1a5b557c service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:27.782938 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6554a11-d401-460a-95e5-18ae1a5b557c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:27.799538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:27.855370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be34446-4e5d-44b9-bd26-b54bd79b652a service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:27.860844 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c5a6e-7216-4e6a-bb04-99e75bd3764b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:27.862127 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c5a6e-7216-4e6a-bb04-99e75bd3764b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:27.862410 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c5a6e-7216-4e6a-bb04-99e75bd3764b admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:27.862685 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c5a6e-7216-4e6a-bb04-99e75bd3764b admin admin] PATH_INFO: `/v3/users/7fc69bc54b4f43bd9ba09cc9b46e11ca` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:27.864306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c5a6e-7216-4e6a-bb04-99e75bd3764b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fc69bc54b4f43bd9ba09cc9b46e11ca)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:27.866693 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c5a6e-7216-4e6a-bb04-99e75bd3764b admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:27.881212 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6554a11-d401-460a-95e5-18ae1a5b557c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:27.882815 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6554a11-d401-460a-95e5-18ae1a5b557c service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:27.904560 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1256/2520] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:31:27.956828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:27.982213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c67c5a6e-7216-4e6a-bb04-99e75bd3764b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fc69bc54b4f43bd9ba09cc9b46e11ca'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:27.995358 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c67c5a6e-7216-4e6a-bb04-99e75bd3764b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fc69bc54b4f43bd9ba09cc9b46e11ca'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:27.996208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c67c5a6e-7216-4e6a-bb04-99e75bd3764b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fc69bc54b4f43bd9ba09cc9b46e11ca'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:28.006791 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c67c5a6e-7216-4e6a-bb04-99e75bd3764b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fc69bc54b4f43bd9ba09cc9b46e11ca'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:28.034383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c67c5a6e-7216-4e6a-bb04-99e75bd3764b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fc69bc54b4f43bd9ba09cc9b46e11ca'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:28.048611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1265/2521] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:31:27 2022] DELETE /identity/v3/users/7fc69bc54b4f43bd9ba09cc9b46e11ca => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:31:28.053046 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff94cb2-cd4a-4b70-b92d-4f9145f8b2dc None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:28.053471 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff94cb2-cd4a-4b70-b92d-4f9145f8b2dc None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:28.053950 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff94cb2-cd4a-4b70-b92d-4f9145f8b2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:28.096063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7ff94cb2-cd4a-4b70-b92d-4f9145f8b2dc None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:28.099031 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7ff94cb2-cd4a-4b70-b92d-4f9145f8b2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:28.249009 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1257/2522] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:28.266134 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c5a6e-7216-4e6a-bb04-99e75bd3764b admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:28.270243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c67c5a6e-7216-4e6a-bb04-99e75bd3764b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:28.374924 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c5a6e-7216-4e6a-bb04-99e75bd3764b admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:28.381936 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eec41e-0822-4691-b28f-474f457f98fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:28.382818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eec41e-0822-4691-b28f-474f457f98fe service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:28.383422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eec41e-0822-4691-b28f-474f457f98fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:28.383931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eec41e-0822-4691-b28f-474f457f98fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:28.404746 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eec41e-0822-4691-b28f-474f457f98fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:28.406685 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eec41e-0822-4691-b28f-474f457f98fe service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:28.420285 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1266/2523] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:31:28.822683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff94cb2-cd4a-4b70-b92d-4f9145f8b2dc None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:28.824360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7ff94cb2-cd4a-4b70-b92d-4f9145f8b2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:28.838969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff94cb2-cd4a-4b70-b92d-4f9145f8b2dc None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:28.845212 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:28.845502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:28.845864 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:28.846321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8 admin admin] PATH_INFO: `/v3/projects/12332bafe0274f9f9afd715805c03742` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:28.848195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12332bafe0274f9f9afd715805c03742)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:28.850658 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:28.921906 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12332bafe0274f9f9afd715805c03742', 'request_id': 'req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:28.928908 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12332bafe0274f9f9afd715805c03742', 'request_id': 'req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:28.942201 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12332bafe0274f9f9afd715805c03742', 'request_id': 'req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:28.961533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8 admin admin] The token cache is being invalidate because project 12332bafe0274f9f9afd715805c03742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:31:28.961533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:28.962673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:28.964595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1258/2524] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:31:28 2022] DELETE /identity/v3/projects/12332bafe0274f9f9afd715805c03742 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:31:28.970872 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eec41e-0822-4691-b28f-474f457f98fe service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:28.986879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eec41e-0822-4691-b28f-474f457f98fe service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:29.070383 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210adb7-ac39-4d1b-b759-f58f19033a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:29.071039 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210adb7-ac39-4d1b-b759-f58f19033a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:29.071343 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210adb7-ac39-4d1b-b759-f58f19033a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:29.071617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210adb7-ac39-4d1b-b759-f58f19033a13 admin admin] PATH_INFO: `/v3/users/d6b08f5f9a6e48be8202afdfcfe502f4` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:29.072814 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210adb7-ac39-4d1b-b759-f58f19033a13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6b08f5f9a6e48be8202afdfcfe502f4)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:29.074818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210adb7-ac39-4d1b-b759-f58f19033a13 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:29.137935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3210adb7-ac39-4d1b-b759-f58f19033a13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6b08f5f9a6e48be8202afdfcfe502f4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:29.149083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3210adb7-ac39-4d1b-b759-f58f19033a13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6b08f5f9a6e48be8202afdfcfe502f4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:29.150289 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3210adb7-ac39-4d1b-b759-f58f19033a13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6b08f5f9a6e48be8202afdfcfe502f4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:29.155869 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:29.164263 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3210adb7-ac39-4d1b-b759-f58f19033a13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6b08f5f9a6e48be8202afdfcfe502f4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:29.180956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-3210adb7-ac39-4d1b-b759-f58f19033a13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6b08f5f9a6e48be8202afdfcfe502f4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:29.187918 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1267/2525] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:31:28 2022] DELETE /identity/v3/users/d6b08f5f9a6e48be8202afdfcfe502f4 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:31:29.925475 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:29.927297 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:29.942897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dda9d1-e448-4c8f-8bfa-ff8b00060ac8 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:30.014355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:30.044183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0222758d-ddf1-4563-9349-61ddf34b8e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:30.045224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222758d-ddf1-4563-9349-61ddf34b8e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:30.045695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222758d-ddf1-4563-9349-61ddf34b8e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:30.046240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0222758d-ddf1-4563-9349-61ddf34b8e51 admin admin] PATH_INFO: `/v3/projects/e136bf4f0a2445a69a1ffeb09198858d` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:30.048608 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222758d-ddf1-4563-9349-61ddf34b8e51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e136bf4f0a2445a69a1ffeb09198858d)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:30.051062 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0222758d-ddf1-4563-9349-61ddf34b8e51 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:30.121408 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0222758d-ddf1-4563-9349-61ddf34b8e51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e136bf4f0a2445a69a1ffeb09198858d', 'request_id': 'req-0222758d-ddf1-4563-9349-61ddf34b8e51'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:30.131083 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0222758d-ddf1-4563-9349-61ddf34b8e51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e136bf4f0a2445a69a1ffeb09198858d', 'request_id': 'req-0222758d-ddf1-4563-9349-61ddf34b8e51'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:30.146049 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0222758d-ddf1-4563-9349-61ddf34b8e51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e136bf4f0a2445a69a1ffeb09198858d', 'request_id': 'req-0222758d-ddf1-4563-9349-61ddf34b8e51'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:30.160636 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0222758d-ddf1-4563-9349-61ddf34b8e51 admin admin] The token cache is being invalidate because project e136bf4f0a2445a69a1ffeb09198858d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:31:30.160932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0222758d-ddf1-4563-9349-61ddf34b8e51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:30.161649 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-0222758d-ddf1-4563-9349-61ddf34b8e51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:30.163574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1259/2526] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:31:29 2022] DELETE /identity/v3/projects/e136bf4f0a2445a69a1ffeb09198858d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:31:30.169327 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee821-1376-4b02-8ff9-2b0333ca1a61 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:30.170260 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee821-1376-4b02-8ff9-2b0333ca1a61 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:30.170260 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee821-1376-4b02-8ff9-2b0333ca1a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:30.193294 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-a26ee821-1376-4b02-8ff9-2b0333ca1a61 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:30.195253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-a26ee821-1376-4b02-8ff9-2b0333ca1a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:30.280915 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1268/2527] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:31:30.289321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355e85e-d708-474f-beea-a4d41ae9147c None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:30.289618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355e85e-d708-474f-beea-a4d41ae9147c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:30.290032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355e85e-d708-474f-beea-a4d41ae9147c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:30.324532 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-1355e85e-d708-474f-beea-a4d41ae9147c None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:30.325986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1355e85e-d708-474f-beea-a4d41ae9147c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:30.339919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1260/2528] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:30.344498 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ee821-1376-4b02-8ff9-2b0333ca1a61 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:30.362573 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ee821-1376-4b02-8ff9-2b0333ca1a61 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:30.371806 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7c6fd-f10c-42e3-a0fd-b3920f7f0576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:30.372824 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7c6fd-f10c-42e3-a0fd-b3920f7f0576 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:30.373291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7c6fd-f10c-42e3-a0fd-b3920f7f0576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:30.373988 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7c6fd-f10c-42e3-a0fd-b3920f7f0576 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:30.375069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7c6fd-f10c-42e3-a0fd-b3920f7f0576 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:31:30.375576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7c6fd-f10c-42e3-a0fd-b3920f7f0576 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:30.379129 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7c6fd-f10c-42e3-a0fd-b3920f7f0576 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:30.402725 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1269/2529] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:31:30.407142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355e85e-d708-474f-beea-a4d41ae9147c None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:30.423897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355e85e-d708-474f-beea-a4d41ae9147c None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:30.432183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20b843-27a1-4867-9d0c-e39f452e27a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:30.433315 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20b843-27a1-4867-9d0c-e39f452e27a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:30.433702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20b843-27a1-4867-9d0c-e39f452e27a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:30.434114 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20b843-27a1-4867-9d0c-e39f452e27a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:30.435196 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20b843-27a1-4867-9d0c-e39f452e27a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:30.439767 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20b843-27a1-4867-9d0c-e39f452e27a4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:30.493915 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1261/2530] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:30 2022] POST /identity/v3/projects => generated 369 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:31:30.499874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7c6fd-f10c-42e3-a0fd-b3920f7f0576 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:30.521100 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7c6fd-f10c-42e3-a0fd-b3920f7f0576 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:30.650600 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c8140-ce86-4dcb-b5c9-dcf2d1a13324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:30.651730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c8140-ce86-4dcb-b5c9-dcf2d1a13324 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:30.652237 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c8140-ce86-4dcb-b5c9-dcf2d1a13324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:30.652970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c8140-ce86-4dcb-b5c9-dcf2d1a13324 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:30.654280 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c8140-ce86-4dcb-b5c9-dcf2d1a13324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:30.657976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c8140-ce86-4dcb-b5c9-dcf2d1a13324 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:30.702890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1270/2531] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:30 2022] POST /identity/v3/users => generated 285 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:30.709417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20b843-27a1-4867-9d0c-e39f452e27a4 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:30.728679 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20b843-27a1-4867-9d0c-e39f452e27a4 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:30.736415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8650729-9b3e-4645-9a48-bf9562276511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:30.737411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8650729-9b3e-4645-9a48-bf9562276511 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:30.737905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8650729-9b3e-4645-9a48-bf9562276511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:30.738377 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8650729-9b3e-4645-9a48-bf9562276511 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:30.739588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8650729-9b3e-4645-9a48-bf9562276511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:30.742749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8650729-9b3e-4645-9a48-bf9562276511 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:30.764052 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1262/2532] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:30.769185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c8140-ce86-4dcb-b5c9-dcf2d1a13324 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:30.784476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c8140-ce86-4dcb-b5c9-dcf2d1a13324 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:30.791610 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618453f6-4caf-4e67-ab8b-3b1d7e51aee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:30.793036 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618453f6-4caf-4e67-ab8b-3b1d7e51aee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:30.793036 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618453f6-4caf-4e67-ab8b-3b1d7e51aee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:30.793036 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618453f6-4caf-4e67-ab8b-3b1d7e51aee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:30.793751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618453f6-4caf-4e67-ab8b-3b1d7e51aee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:30.795676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618453f6-4caf-4e67-ab8b-3b1d7e51aee3 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:30.833485 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1271/2533] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:30.834734 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8650729-9b3e-4645-9a48-bf9562276511 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:30.850221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8650729-9b3e-4645-9a48-bf9562276511 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:30.855970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d7cc7-153f-41a7-9d60-92e98ea83ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:30.856815 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d7cc7-153f-41a7-9d60-92e98ea83ece admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:30.857098 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d7cc7-153f-41a7-9d60-92e98ea83ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:30.857440 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d7cc7-153f-41a7-9d60-92e98ea83ece admin admin] PATH_INFO: `/v3/projects/b1e48393e4bb4bb48ca444f86a07d29a/users/688d05a575ff4179a0344545c57c3cdb/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:30.904409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d7cc7-153f-41a7-9d60-92e98ea83ece admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1e48393e4bb4bb48ca444f86a07d29a, user_id=688d05a575ff4179a0344545c57c3cdb, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:30.910474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d7cc7-153f-41a7-9d60-92e98ea83ece admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:30.955195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1263/2534] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:30 2022] PUT /identity/v3/projects/b1e48393e4bb4bb48ca444f86a07d29a/users/688d05a575ff4179a0344545c57c3cdb/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:31:30.974940 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b6d31-1aaf-4c21-80c3-c3fa9690a11a None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:30.976076 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b6d31-1aaf-4c21-80c3-c3fa9690a11a None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:30.976623 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b6d31-1aaf-4c21-80c3-c3fa9690a11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:31.039932 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-644b6d31-1aaf-4c21-80c3-c3fa9690a11a None None] MFA Rules not processed for user `688d05a575ff4179a0344545c57c3cdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:31.042051 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-644b6d31-1aaf-4c21-80c3-c3fa9690a11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:31.103704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:31.158649 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1272/2535] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:30 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:31:31.177996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d7cc7-153f-41a7-9d60-92e98ea83ece admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:31.180888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-870d7cc7-153f-41a7-9d60-92e98ea83ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:31.286152 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d7cc7-153f-41a7-9d60-92e98ea83ece admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:31.293390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c202a3b-faf3-4f91-bf69-cbe620c5b239 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f696dfe4b34c4aa19104b5f7ee6e70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:31.294471 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c202a3b-faf3-4f91-bf69-cbe620c5b239 service glance] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:31.294946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c202a3b-faf3-4f91-bf69-cbe620c5b239 service glance] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:31.295416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c202a3b-faf3-4f91-bf69-cbe620c5b239 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:31.310409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c202a3b-faf3-4f91-bf69-cbe620c5b239 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:31.313290 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c202a3b-faf3-4f91-bf69-cbe620c5b239 service glance] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:31.332135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1264/2536] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:31 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:39.150082 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b6d31-1aaf-4c21-80c3-c3fa9690a11a None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:39.151861 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-644b6d31-1aaf-4c21-80c3-c3fa9690a11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:39.258535 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b6d31-1aaf-4c21-80c3-c3fa9690a11a None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:39.263693 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ac58b-09fb-4f90-9e4d-ff0706b7c5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:39.264451 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ac58b-09fb-4f90-9e4d-ff0706b7c5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:39.265249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ac58b-09fb-4f90-9e4d-ff0706b7c5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:39.265249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ac58b-09fb-4f90-9e4d-ff0706b7c5c8 admin admin] PATH_INFO: `/v3/users/7d1ce5135c64437cb48b1e1c14cfec6c` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:39.266684 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ac58b-09fb-4f90-9e4d-ff0706b7c5c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d1ce5135c64437cb48b1e1c14cfec6c)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:39.269345 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ac58b-09fb-4f90-9e4d-ff0706b7c5c8 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:39.374409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-2f6ac58b-09fb-4f90-9e4d-ff0706b7c5c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d1ce5135c64437cb48b1e1c14cfec6c'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:39.394638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-2f6ac58b-09fb-4f90-9e4d-ff0706b7c5c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d1ce5135c64437cb48b1e1c14cfec6c'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:39.395536 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-2f6ac58b-09fb-4f90-9e4d-ff0706b7c5c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d1ce5135c64437cb48b1e1c14cfec6c'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:39.405611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-2f6ac58b-09fb-4f90-9e4d-ff0706b7c5c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d1ce5135c64437cb48b1e1c14cfec6c'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:39.428869 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-2f6ac58b-09fb-4f90-9e4d-ff0706b7c5c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d1ce5135c64437cb48b1e1c14cfec6c'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:39.454191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1273/2537] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:31:39 2022] DELETE /identity/v3/users/7d1ce5135c64437cb48b1e1c14cfec6c => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:39.459873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48f65e-2a8a-440e-921a-9046c24372ec None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:39.461617 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48f65e-2a8a-440e-921a-9046c24372ec None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:39.462786 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48f65e-2a8a-440e-921a-9046c24372ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:39.496250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4d48f65e-2a8a-440e-921a-9046c24372ec None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:39.498290 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4d48f65e-2a8a-440e-921a-9046c24372ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:39.507238 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:39.632209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1265/2538] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:31:39.659196 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ac58b-09fb-4f90-9e4d-ff0706b7c5c8 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:39.660980 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-2f6ac58b-09fb-4f90-9e4d-ff0706b7c5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:39.768116 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ac58b-09fb-4f90-9e4d-ff0706b7c5c8 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:39.773498 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255fe82-bb67-4b92-894e-cae082ce472e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:39.774787 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255fe82-bb67-4b92-894e-cae082ce472e service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:39.775347 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255fe82-bb67-4b92-894e-cae082ce472e service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:39.775964 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255fe82-bb67-4b92-894e-cae082ce472e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:39.790143 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:39.805358 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255fe82-bb67-4b92-894e-cae082ce472e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:39.808642 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255fe82-bb67-4b92-894e-cae082ce472e service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:39.821041 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1274/2539] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:40.281842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48f65e-2a8a-440e-921a-9046c24372ec None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:40.283538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4d48f65e-2a8a-440e-921a-9046c24372ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:40.310236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48f65e-2a8a-440e-921a-9046c24372ec None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:40.329482 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:40.330639 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:40.337938 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:40.338443 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6 admin admin] PATH_INFO: `/v3/projects/4896130f6b08478ba5c6dff0379414c0` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:40.339879 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4896130f6b08478ba5c6dff0379414c0)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:40.343556 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:40.455339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4896130f6b08478ba5c6dff0379414c0', 'request_id': 'req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:40.471587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4896130f6b08478ba5c6dff0379414c0', 'request_id': 'req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:40.487690 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4896130f6b08478ba5c6dff0379414c0', 'request_id': 'req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:40.526186 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6 admin admin] The token cache is being invalidate because project 4896130f6b08478ba5c6dff0379414c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:31:40.526906 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:40.528592 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:40.531397 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1266/2540] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:31:40 2022] DELETE /identity/v3/projects/4896130f6b08478ba5c6dff0379414c0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:40.536653 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255fe82-bb67-4b92-894e-cae082ce472e service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:40.547627 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255fe82-bb67-4b92-894e-cae082ce472e service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:40.643661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18c949-2e6d-4916-9c07-d419fb0f1e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:40.644573 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18c949-2e6d-4916-9c07-d419fb0f1e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:40.644852 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18c949-2e6d-4916-9c07-d419fb0f1e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:40.645220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18c949-2e6d-4916-9c07-d419fb0f1e60 admin admin] PATH_INFO: `/v3/users/fe9920820c314477beeda6e9aea4da2e` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:40.646914 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18c949-2e6d-4916-9c07-d419fb0f1e60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe9920820c314477beeda6e9aea4da2e)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:40.649305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18c949-2e6d-4916-9c07-d419fb0f1e60 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:40.735744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8a18c949-2e6d-4916-9c07-d419fb0f1e60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe9920820c314477beeda6e9aea4da2e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:40.751377 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8a18c949-2e6d-4916-9c07-d419fb0f1e60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe9920820c314477beeda6e9aea4da2e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:40.752585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8a18c949-2e6d-4916-9c07-d419fb0f1e60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe9920820c314477beeda6e9aea4da2e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:40.777865 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8a18c949-2e6d-4916-9c07-d419fb0f1e60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe9920820c314477beeda6e9aea4da2e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:40.801838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8a18c949-2e6d-4916-9c07-d419fb0f1e60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe9920820c314477beeda6e9aea4da2e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:40.811913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1275/2541] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:31:40 2022] DELETE /identity/v3/users/fe9920820c314477beeda6e9aea4da2e => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:41.312729 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:41.314337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:41.456407 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4e3f7-7973-4916-b6b1-3e12c3e596c6 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:41.490520 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:41.540867 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b989487-e1b8-46d0-902c-9c62aaf39286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:41.541708 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b989487-e1b8-46d0-902c-9c62aaf39286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:41.542078 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b989487-e1b8-46d0-902c-9c62aaf39286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:41.542589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b989487-e1b8-46d0-902c-9c62aaf39286 admin admin] PATH_INFO: `/v3/projects/ef3dfc6bd9b7439d9b39ceae75b5f588` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:41.544125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b989487-e1b8-46d0-902c-9c62aaf39286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef3dfc6bd9b7439d9b39ceae75b5f588)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:41.546520 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b989487-e1b8-46d0-902c-9c62aaf39286 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:41.642988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1b989487-e1b8-46d0-902c-9c62aaf39286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef3dfc6bd9b7439d9b39ceae75b5f588', 'request_id': 'req-1b989487-e1b8-46d0-902c-9c62aaf39286'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:41.653917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1b989487-e1b8-46d0-902c-9c62aaf39286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef3dfc6bd9b7439d9b39ceae75b5f588', 'request_id': 'req-1b989487-e1b8-46d0-902c-9c62aaf39286'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:41.669990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1b989487-e1b8-46d0-902c-9c62aaf39286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef3dfc6bd9b7439d9b39ceae75b5f588', 'request_id': 'req-1b989487-e1b8-46d0-902c-9c62aaf39286'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:41.694587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1b989487-e1b8-46d0-902c-9c62aaf39286 admin admin] The token cache is being invalidate because project ef3dfc6bd9b7439d9b39ceae75b5f588 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:31:41.694587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1b989487-e1b8-46d0-902c-9c62aaf39286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:41.694587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1b989487-e1b8-46d0-902c-9c62aaf39286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:41.694587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1267/2542] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:31:41 2022] DELETE /identity/v3/projects/ef3dfc6bd9b7439d9b39ceae75b5f588 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:41.703923 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b63369-ac4a-4817-9d95-83267cda8350 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:41.706341 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b63369-ac4a-4817-9d95-83267cda8350 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:41.708236 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b63369-ac4a-4817-9d95-83267cda8350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:41.839425 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-12b63369-ac4a-4817-9d95-83267cda8350 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:41.841015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-12b63369-ac4a-4817-9d95-83267cda8350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:41.913397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:41.962330 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1276/2543] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:41.971265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987dbc0-31a4-4ffc-aa87-13cee0db82b8 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:41.971561 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987dbc0-31a4-4ffc-aa87-13cee0db82b8 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:41.971841 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987dbc0-31a4-4ffc-aa87-13cee0db82b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:41.998898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-5987dbc0-31a4-4ffc-aa87-13cee0db82b8 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:42.000149 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5987dbc0-31a4-4ffc-aa87-13cee0db82b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:42.016374 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1268/2544] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:31:42.022505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b63369-ac4a-4817-9d95-83267cda8350 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:42.036458 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b63369-ac4a-4817-9d95-83267cda8350 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:42.043890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d4603-fbf4-46c0-96be-5be8ade3b35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:42.044868 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d4603-fbf4-46c0-96be-5be8ade3b35a admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:42.045160 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d4603-fbf4-46c0-96be-5be8ade3b35a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:42.045615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d4603-fbf4-46c0-96be-5be8ade3b35a admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:42.046961 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d4603-fbf4-46c0-96be-5be8ade3b35a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:31:42.047634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d4603-fbf4-46c0-96be-5be8ade3b35a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:42.050655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d4603-fbf4-46c0-96be-5be8ade3b35a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:42.068719 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1277/2545] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:31:42.074324 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987dbc0-31a4-4ffc-aa87-13cee0db82b8 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:42.086401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987dbc0-31a4-4ffc-aa87-13cee0db82b8 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:42.091666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcda6a-9697-46f3-899a-6694b08b9689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:42.097210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcda6a-9697-46f3-899a-6694b08b9689 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:42.097210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcda6a-9697-46f3-899a-6694b08b9689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:42.097210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcda6a-9697-46f3-899a-6694b08b9689 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:42.097210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fcda6a-9697-46f3-899a-6694b08b9689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:42.098413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fcda6a-9697-46f3-899a-6694b08b9689 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:42.134266 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1269/2546] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:42 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:42.138861 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d4603-fbf4-46c0-96be-5be8ade3b35a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:42.152165 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d4603-fbf4-46c0-96be-5be8ade3b35a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:42.251185 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226a93b-252f-48bd-8fda-f63c443cfac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:42.251831 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226a93b-252f-48bd-8fda-f63c443cfac2 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:42.252106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226a93b-252f-48bd-8fda-f63c443cfac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:42.252370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226a93b-252f-48bd-8fda-f63c443cfac2 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:42.253055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226a93b-252f-48bd-8fda-f63c443cfac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:42.255046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226a93b-252f-48bd-8fda-f63c443cfac2 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:42.286660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1278/2547] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:42 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:31:42.292917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcda6a-9697-46f3-899a-6694b08b9689 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:42.304957 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcda6a-9697-46f3-899a-6694b08b9689 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:42.311021 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4af6d-26bc-4259-891c-ab720fe28f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:42.311864 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4af6d-26bc-4259-891c-ab720fe28f70 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:42.312682 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4af6d-26bc-4259-891c-ab720fe28f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:42.313029 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4af6d-26bc-4259-891c-ab720fe28f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:42.313869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4af6d-26bc-4259-891c-ab720fe28f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:42.315562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4af6d-26bc-4259-891c-ab720fe28f70 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:42.331516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1270/2548] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:31:42.331743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226a93b-252f-48bd-8fda-f63c443cfac2 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:42.347077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226a93b-252f-48bd-8fda-f63c443cfac2 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:42.354052 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25071e2e-2a8e-4d56-bc02-7d5cee92381c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:42.354749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25071e2e-2a8e-4d56-bc02-7d5cee92381c admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:42.355022 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25071e2e-2a8e-4d56-bc02-7d5cee92381c admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:42.355289 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25071e2e-2a8e-4d56-bc02-7d5cee92381c admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:42.356220 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25071e2e-2a8e-4d56-bc02-7d5cee92381c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:42.358402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25071e2e-2a8e-4d56-bc02-7d5cee92381c admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:42.373271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1279/2549] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:42.378104 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4af6d-26bc-4259-891c-ab720fe28f70 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:42.389254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4af6d-26bc-4259-891c-ab720fe28f70 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:42.394554 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1111dfd-4df5-424a-890c-1f6dd22fd95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:42.395246 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1111dfd-4df5-424a-890c-1f6dd22fd95a admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:42.395520 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1111dfd-4df5-424a-890c-1f6dd22fd95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:42.395788 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1111dfd-4df5-424a-890c-1f6dd22fd95a admin admin] PATH_INFO: `/v3/projects/19ee08f33edb44418560db4153a19c19/users/b5f32582315843d596ca643bfac8fae4/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:42.414765 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1111dfd-4df5-424a-890c-1f6dd22fd95a admin admin] RBAC: Authorizing `identity:create_grant(project_id=19ee08f33edb44418560db4153a19c19, user_id=b5f32582315843d596ca643bfac8fae4, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:42.417687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1111dfd-4df5-424a-890c-1f6dd22fd95a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:42.437891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1271/2550] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:42 2022] PUT /identity/v3/projects/19ee08f33edb44418560db4153a19c19/users/b5f32582315843d596ca643bfac8fae4/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:42.451424 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456d8ab-cce9-4b8c-9b8f-0222b6f6b76d None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:42.451865 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456d8ab-cce9-4b8c-9b8f-0222b6f6b76d None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:42.452314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456d8ab-cce9-4b8c-9b8f-0222b6f6b76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:42.503927 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-b456d8ab-cce9-4b8c-9b8f-0222b6f6b76d None None] MFA Rules not processed for user `b5f32582315843d596ca643bfac8fae4`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:42.503927 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-b456d8ab-cce9-4b8c-9b8f-0222b6f6b76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:42.610591 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1280/2551] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:42.619819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1111dfd-4df5-424a-890c-1f6dd22fd95a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:42.716051 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:42.720171 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1111dfd-4df5-424a-890c-1f6dd22fd95a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:42.728728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45a95e-a44b-4f26-bad9-4fe852a7c45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:42.729893 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45a95e-a44b-4f26-bad9-4fe852a7c45d admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:42.730540 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45a95e-a44b-4f26-bad9-4fe852a7c45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:42.731010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45a95e-a44b-4f26-bad9-4fe852a7c45d admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:42.732172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45a95e-a44b-4f26-bad9-4fe852a7c45d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:42.735567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45a95e-a44b-4f26-bad9-4fe852a7c45d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:42.773095 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1272/2552] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:42 2022] POST /identity/v3/projects => generated 371 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:31:42.777943 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456d8ab-cce9-4b8c-9b8f-0222b6f6b76d None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:42.790291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456d8ab-cce9-4b8c-9b8f-0222b6f6b76d None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:42.832825 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:42.865372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077ce8c-f0c7-4015-95fa-c512ff0ed171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:42.866039 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077ce8c-f0c7-4015-95fa-c512ff0ed171 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:42.866421 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077ce8c-f0c7-4015-95fa-c512ff0ed171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:42.866756 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077ce8c-f0c7-4015-95fa-c512ff0ed171 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:42.867432 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077ce8c-f0c7-4015-95fa-c512ff0ed171 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:42.870214 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077ce8c-f0c7-4015-95fa-c512ff0ed171 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:42.905168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1281/2553] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:42 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:42.912097 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45a95e-a44b-4f26-bad9-4fe852a7c45d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:42.939708 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45a95e-a44b-4f26-bad9-4fe852a7c45d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:42.946588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065e905-547a-49a9-825b-fa0c57263085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:42.947434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065e905-547a-49a9-825b-fa0c57263085 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:42.948044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065e905-547a-49a9-825b-fa0c57263085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:42.948238 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065e905-547a-49a9-825b-fa0c57263085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:42.949489 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065e905-547a-49a9-825b-fa0c57263085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:42.951999 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065e905-547a-49a9-825b-fa0c57263085 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:42.977712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1273/2554] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:42.984652 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077ce8c-f0c7-4015-95fa-c512ff0ed171 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:43.010252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077ce8c-f0c7-4015-95fa-c512ff0ed171 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:43.020021 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d969ee2-8e13-4291-aefa-afdd946f9a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:43.021339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d969ee2-8e13-4291-aefa-afdd946f9a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:43.021720 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d969ee2-8e13-4291-aefa-afdd946f9a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:43.022241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d969ee2-8e13-4291-aefa-afdd946f9a2a admin admin] PATH_INFO: `/v3/projects/aaf4e8e46038428dbac0b78c0c85bdff/users/b82fc3015fe24bec93797afedd55b67e/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:43.044450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d969ee2-8e13-4291-aefa-afdd946f9a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaf4e8e46038428dbac0b78c0c85bdff, user_id=b82fc3015fe24bec93797afedd55b67e, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:43.047310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d969ee2-8e13-4291-aefa-afdd946f9a2a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:43.063018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1282/2555] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:42 2022] PUT /identity/v3/projects/aaf4e8e46038428dbac0b78c0c85bdff/users/b82fc3015fe24bec93797afedd55b67e/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:31:43.069184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065e905-547a-49a9-825b-fa0c57263085 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:43.087450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065e905-547a-49a9-825b-fa0c57263085 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:43.157712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9a592-bb98-446b-a497-474d1d283290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:43.158498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9a592-bb98-446b-a497-474d1d283290 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:43.158777 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9a592-bb98-446b-a497-474d1d283290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:43.159053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9a592-bb98-446b-a497-474d1d283290 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:43.159757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9a592-bb98-446b-a497-474d1d283290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:43.161391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9a592-bb98-446b-a497-474d1d283290 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:43.172323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1274/2556] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:31:43.176731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d969ee2-8e13-4291-aefa-afdd946f9a2a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:43.187951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d969ee2-8e13-4291-aefa-afdd946f9a2a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:43.193684 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69da463-ae67-4e60-bfd5-ca0b27f7427c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:43.194486 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69da463-ae67-4e60-bfd5-ca0b27f7427c admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:43.195032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69da463-ae67-4e60-bfd5-ca0b27f7427c admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:43.195557 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69da463-ae67-4e60-bfd5-ca0b27f7427c admin admin] PATH_INFO: `/v3/projects/aaf4e8e46038428dbac0b78c0c85bdff/users/b82fc3015fe24bec93797afedd55b67e/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:43.199567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69da463-ae67-4e60-bfd5-ca0b27f7427c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaf4e8e46038428dbac0b78c0c85bdff, user_id=b82fc3015fe24bec93797afedd55b67e, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:43.207423 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69da463-ae67-4e60-bfd5-ca0b27f7427c admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:43.227313 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1283/2557] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:43 2022] PUT /identity/v3/projects/aaf4e8e46038428dbac0b78c0c85bdff/users/b82fc3015fe24bec93797afedd55b67e/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:43.243911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc486cd-01d6-44c0-a872-4a888a00b037 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:43.244395 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc486cd-01d6-44c0-a872-4a888a00b037 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:43.244817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc486cd-01d6-44c0-a872-4a888a00b037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:43.280671 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-8dc486cd-01d6-44c0-a872-4a888a00b037 None None] MFA Rules not processed for user `b82fc3015fe24bec93797afedd55b67e`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:43.282173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8dc486cd-01d6-44c0-a872-4a888a00b037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:43.382945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1275/2558] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:43.396588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69da463-ae67-4e60-bfd5-ca0b27f7427c admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:43.398590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d69da463-ae67-4e60-bfd5-ca0b27f7427c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:43.505672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69da463-ae67-4e60-bfd5-ca0b27f7427c admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:43.508892 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8d4da-debf-487e-ac59-27880bd70b72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:43.509643 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8d4da-debf-487e-ac59-27880bd70b72 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:43.509955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8d4da-debf-487e-ac59-27880bd70b72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:43.510234 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8d4da-debf-487e-ac59-27880bd70b72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:43.520860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8d4da-debf-487e-ac59-27880bd70b72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:43.522394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8d4da-debf-487e-ac59-27880bd70b72 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:43.534493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1284/2559] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:43 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:43.863423 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc486cd-01d6-44c0-a872-4a888a00b037 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:43.871609 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8dc486cd-01d6-44c0-a872-4a888a00b037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:43.963977 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc486cd-01d6-44c0-a872-4a888a00b037 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:43.968945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbf2c7-0fd4-4fca-befb-654f643054ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:43.969681 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbf2c7-0fd4-4fca-befb-654f643054ea service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:43.970015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbf2c7-0fd4-4fca-befb-654f643054ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:43.970285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbf2c7-0fd4-4fca-befb-654f643054ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:43.972152 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1cfbf2c7-0fd4-4fca-befb-654f643054ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:43.984456 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:44.095847 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbf2c7-0fd4-4fca-befb-654f643054ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:44.097445 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbf2c7-0fd4-4fca-befb-654f643054ea service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:44.100274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8d4da-debf-487e-ac59-27880bd70b72 service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:44.119096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1276/2560] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:44.135582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8d4da-debf-487e-ac59-27880bd70b72 service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:44.140560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a86745-40c5-4b5b-960d-a68bf4208afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:44.141230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a86745-40c5-4b5b-960d-a68bf4208afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:44.141509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a86745-40c5-4b5b-960d-a68bf4208afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:44.141800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a86745-40c5-4b5b-960d-a68bf4208afa admin admin] PATH_INFO: `/v3/users/b5f32582315843d596ca643bfac8fae4` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:44.143091 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a86745-40c5-4b5b-960d-a68bf4208afa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5f32582315843d596ca643bfac8fae4)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:44.145060 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a86745-40c5-4b5b-960d-a68bf4208afa admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:44.230399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-34a86745-40c5-4b5b-960d-a68bf4208afa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5f32582315843d596ca643bfac8fae4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:44.248025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-34a86745-40c5-4b5b-960d-a68bf4208afa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5f32582315843d596ca643bfac8fae4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:44.249055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-34a86745-40c5-4b5b-960d-a68bf4208afa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5f32582315843d596ca643bfac8fae4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:44.268423 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-34a86745-40c5-4b5b-960d-a68bf4208afa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5f32582315843d596ca643bfac8fae4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:44.283976 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:44.295403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-34a86745-40c5-4b5b-960d-a68bf4208afa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5f32582315843d596ca643bfac8fae4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:44.311897 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1285/2561] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:31:44 2022] DELETE /identity/v3/users/b5f32582315843d596ca643bfac8fae4 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:31:44.317828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde894a-38f1-4d29-9114-5b274dc8c053 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:44.318271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde894a-38f1-4d29-9114-5b274dc8c053 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:44.318511 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde894a-38f1-4d29-9114-5b274dc8c053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:44.348405 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-0cde894a-38f1-4d29-9114-5b274dc8c053 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:44.352235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0cde894a-38f1-4d29-9114-5b274dc8c053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:44.458067 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1277/2562] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:31:44.468250 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a86745-40c5-4b5b-960d-a68bf4208afa admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:44.470063 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-34a86745-40c5-4b5b-960d-a68bf4208afa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:44.590961 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a86745-40c5-4b5b-960d-a68bf4208afa admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:44.597312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28cfd9-a905-44ab-ae7f-69ffe3851218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:44.598715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28cfd9-a905-44ab-ae7f-69ffe3851218 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:44.598715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28cfd9-a905-44ab-ae7f-69ffe3851218 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:44.599051 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c28cfd9-a905-44ab-ae7f-69ffe3851218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:44.613947 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28cfd9-a905-44ab-ae7f-69ffe3851218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:44.615515 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c28cfd9-a905-44ab-ae7f-69ffe3851218 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:44.648496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1286/2563] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:45.093751 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde894a-38f1-4d29-9114-5b274dc8c053 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:45.096121 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-0cde894a-38f1-4d29-9114-5b274dc8c053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:45.118462 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde894a-38f1-4d29-9114-5b274dc8c053 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:45.127329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b182b-7d21-4b87-9334-bffa2cc22d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:45.127843 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b182b-7d21-4b87-9334-bffa2cc22d57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:45.128466 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b182b-7d21-4b87-9334-bffa2cc22d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:45.128728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b182b-7d21-4b87-9334-bffa2cc22d57 admin admin] PATH_INFO: `/v3/projects/19ee08f33edb44418560db4153a19c19` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:45.131277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b182b-7d21-4b87-9334-bffa2cc22d57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19ee08f33edb44418560db4153a19c19)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:45.135349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b182b-7d21-4b87-9334-bffa2cc22d57 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:45.207866 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-110b182b-7d21-4b87-9334-bffa2cc22d57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19ee08f33edb44418560db4153a19c19', 'request_id': 'req-110b182b-7d21-4b87-9334-bffa2cc22d57'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:45.214727 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-110b182b-7d21-4b87-9334-bffa2cc22d57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19ee08f33edb44418560db4153a19c19', 'request_id': 'req-110b182b-7d21-4b87-9334-bffa2cc22d57'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:45.223675 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-110b182b-7d21-4b87-9334-bffa2cc22d57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19ee08f33edb44418560db4153a19c19', 'request_id': 'req-110b182b-7d21-4b87-9334-bffa2cc22d57'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:45.240067 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-110b182b-7d21-4b87-9334-bffa2cc22d57 admin admin] The token cache is being invalidate because project 19ee08f33edb44418560db4153a19c19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:31:45.240527 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-110b182b-7d21-4b87-9334-bffa2cc22d57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:45.241583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-110b182b-7d21-4b87-9334-bffa2cc22d57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:45.244516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1278/2564] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:31:45 2022] DELETE /identity/v3/projects/19ee08f33edb44418560db4153a19c19 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:31:45.250565 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28cfd9-a905-44ab-ae7f-69ffe3851218 service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:45.267001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c28cfd9-a905-44ab-ae7f-69ffe3851218 service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:45.355320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47dda3-d941-4cf5-a28e-3e2de9ecda85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:45.356113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47dda3-d941-4cf5-a28e-3e2de9ecda85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:45.356413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47dda3-d941-4cf5-a28e-3e2de9ecda85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:45.356690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47dda3-d941-4cf5-a28e-3e2de9ecda85 admin admin] PATH_INFO: `/v3/users/b82fc3015fe24bec93797afedd55b67e` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:45.358085 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47dda3-d941-4cf5-a28e-3e2de9ecda85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b82fc3015fe24bec93797afedd55b67e)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:45.360494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47dda3-d941-4cf5-a28e-3e2de9ecda85 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:45.454154 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-7e47dda3-d941-4cf5-a28e-3e2de9ecda85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b82fc3015fe24bec93797afedd55b67e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:45.463480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-7e47dda3-d941-4cf5-a28e-3e2de9ecda85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b82fc3015fe24bec93797afedd55b67e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:45.464400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-7e47dda3-d941-4cf5-a28e-3e2de9ecda85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b82fc3015fe24bec93797afedd55b67e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:45.476774 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-7e47dda3-d941-4cf5-a28e-3e2de9ecda85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b82fc3015fe24bec93797afedd55b67e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:45.499501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-7e47dda3-d941-4cf5-a28e-3e2de9ecda85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b82fc3015fe24bec93797afedd55b67e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:45.507463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1287/2565] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:31:45 2022] DELETE /identity/v3/users/b82fc3015fe24bec93797afedd55b67e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:46.129317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b182b-7d21-4b87-9334-bffa2cc22d57 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:46.131058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-110b182b-7d21-4b87-9334-bffa2cc22d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:46.145166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b182b-7d21-4b87-9334-bffa2cc22d57 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:46.172911 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:46.231300 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad099f4-f0fe-4e91-9e60-988099cabbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:46.232149 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad099f4-f0fe-4e91-9e60-988099cabbad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:46.232541 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad099f4-f0fe-4e91-9e60-988099cabbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:46.232909 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad099f4-f0fe-4e91-9e60-988099cabbad admin admin] PATH_INFO: `/v3/projects/aaf4e8e46038428dbac0b78c0c85bdff` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:46.235173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad099f4-f0fe-4e91-9e60-988099cabbad admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaf4e8e46038428dbac0b78c0c85bdff)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:46.249825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad099f4-f0fe-4e91-9e60-988099cabbad admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:46.292458 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3ad099f4-f0fe-4e91-9e60-988099cabbad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaf4e8e46038428dbac0b78c0c85bdff', 'request_id': 'req-3ad099f4-f0fe-4e91-9e60-988099cabbad'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:46.297909 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3ad099f4-f0fe-4e91-9e60-988099cabbad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaf4e8e46038428dbac0b78c0c85bdff', 'request_id': 'req-3ad099f4-f0fe-4e91-9e60-988099cabbad'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:46.309196 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3ad099f4-f0fe-4e91-9e60-988099cabbad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaf4e8e46038428dbac0b78c0c85bdff', 'request_id': 'req-3ad099f4-f0fe-4e91-9e60-988099cabbad'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:46.321239 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3ad099f4-f0fe-4e91-9e60-988099cabbad admin admin] The token cache is being invalidate because project aaf4e8e46038428dbac0b78c0c85bdff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:31:46.321523 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3ad099f4-f0fe-4e91-9e60-988099cabbad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:46.322177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3ad099f4-f0fe-4e91-9e60-988099cabbad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:46.323917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1279/2566] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:31:46 2022] DELETE /identity/v3/projects/aaf4e8e46038428dbac0b78c0c85bdff => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:46.329588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcdae7-2fbb-4a22-9887-a19da4af9461 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:46.330143 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcdae7-2fbb-4a22-9887-a19da4af9461 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:46.330250 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dcdae7-2fbb-4a22-9887-a19da4af9461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:46.351780 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-04dcdae7-2fbb-4a22-9887-a19da4af9461 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:46.353360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-04dcdae7-2fbb-4a22-9887-a19da4af9461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:46.358951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:46.452939 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1288/2567] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:46.461651 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a970d0-957c-4530-acea-acb69d249497 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:46.462045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a970d0-957c-4530-acea-acb69d249497 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:46.462249 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a970d0-957c-4530-acea-acb69d249497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:46.489101 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-b1a970d0-957c-4530-acea-acb69d249497 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:46.490443 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b1a970d0-957c-4530-acea-acb69d249497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:46.501141 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1280/2568] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:46.505287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcdae7-2fbb-4a22-9887-a19da4af9461 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:46.516283 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dcdae7-2fbb-4a22-9887-a19da4af9461 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:46.521946 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffbdf3-876b-4f12-a30d-0e66a0e93654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:46.523069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffbdf3-876b-4f12-a30d-0e66a0e93654 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:46.523510 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffbdf3-876b-4f12-a30d-0e66a0e93654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:46.523960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffbdf3-876b-4f12-a30d-0e66a0e93654 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:46.524845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffbdf3-876b-4f12-a30d-0e66a0e93654 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:31:46.525316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffbdf3-876b-4f12-a30d-0e66a0e93654 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:46.527198 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffbdf3-876b-4f12-a30d-0e66a0e93654 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:46.548006 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1289/2569] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:31:46.550531 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a970d0-957c-4530-acea-acb69d249497 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:46.561047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a970d0-957c-4530-acea-acb69d249497 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:46.566945 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16ecfb-7897-4199-9767-2ce6ac96b658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:46.567285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16ecfb-7897-4199-9767-2ce6ac96b658 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:46.567564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16ecfb-7897-4199-9767-2ce6ac96b658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:46.567886 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16ecfb-7897-4199-9767-2ce6ac96b658 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:46.568543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16ecfb-7897-4199-9767-2ce6ac96b658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:46.570557 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16ecfb-7897-4199-9767-2ce6ac96b658 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:46.608574 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1281/2570] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:46 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:46.613584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffbdf3-876b-4f12-a30d-0e66a0e93654 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:46.626603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffbdf3-876b-4f12-a30d-0e66a0e93654 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:46.694724 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec076186-ae48-4a51-9ffd-0c9062e6e25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:46.695545 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec076186-ae48-4a51-9ffd-0c9062e6e25c admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:46.695918 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec076186-ae48-4a51-9ffd-0c9062e6e25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:46.696287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec076186-ae48-4a51-9ffd-0c9062e6e25c admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:46.697065 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec076186-ae48-4a51-9ffd-0c9062e6e25c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:46.701126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec076186-ae48-4a51-9ffd-0c9062e6e25c admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:46.733256 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1290/2571] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:46 2022] POST /identity/v3/users => generated 282 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:46.738147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16ecfb-7897-4199-9767-2ce6ac96b658 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:46.752101 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16ecfb-7897-4199-9767-2ce6ac96b658 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:46.759186 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626f082-82a7-4cc7-b771-48302793cd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:46.760305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626f082-82a7-4cc7-b771-48302793cd3a admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:46.760773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626f082-82a7-4cc7-b771-48302793cd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:46.761207 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626f082-82a7-4cc7-b771-48302793cd3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:46.762149 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626f082-82a7-4cc7-b771-48302793cd3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:46.764482 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626f082-82a7-4cc7-b771-48302793cd3a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:46.780992 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1282/2572] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:46.783274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec076186-ae48-4a51-9ffd-0c9062e6e25c admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:46.794450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec076186-ae48-4a51-9ffd-0c9062e6e25c admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:46.800691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc15626-d6f1-4375-9246-91c50659e403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:46.801345 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc15626-d6f1-4375-9246-91c50659e403 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:46.801688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc15626-d6f1-4375-9246-91c50659e403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:46.802427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc15626-d6f1-4375-9246-91c50659e403 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:46.803779 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc15626-d6f1-4375-9246-91c50659e403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:46.807645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc15626-d6f1-4375-9246-91c50659e403 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:46.825250 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1291/2573] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:46.830708 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626f082-82a7-4cc7-b771-48302793cd3a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:46.842291 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626f082-82a7-4cc7-b771-48302793cd3a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:46.848973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7133991-1190-49c3-aa13-f947897ae7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:46.850351 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7133991-1190-49c3-aa13-f947897ae7ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:46.850351 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7133991-1190-49c3-aa13-f947897ae7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:46.850351 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7133991-1190-49c3-aa13-f947897ae7ca admin admin] PATH_INFO: `/v3/projects/2942f199b22044a099a0160aa37197f6/users/21129074968d40ff8f858401b8de755b/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:46.873253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7133991-1190-49c3-aa13-f947897ae7ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=2942f199b22044a099a0160aa37197f6, user_id=21129074968d40ff8f858401b8de755b, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:46.880307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7133991-1190-49c3-aa13-f947897ae7ca admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:46.901073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1283/2574] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:46 2022] PUT /identity/v3/projects/2942f199b22044a099a0160aa37197f6/users/21129074968d40ff8f858401b8de755b/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:31:46.913928 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d498c5-915c-47f1-8c31-0cafe8d3f1d3 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:46.914195 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d498c5-915c-47f1-8c31-0cafe8d3f1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:46.914475 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d498c5-915c-47f1-8c31-0cafe8d3f1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:46.957321 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-b2d498c5-915c-47f1-8c31-0cafe8d3f1d3 None None] MFA Rules not processed for user `21129074968d40ff8f858401b8de755b`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:46.958671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-b2d498c5-915c-47f1-8c31-0cafe8d3f1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:47.050032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1292/2575] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:46 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:47.054802 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7133991-1190-49c3-aa13-f947897ae7ca admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:47.122155 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:47.140096 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7133991-1190-49c3-aa13-f947897ae7ca admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:47.147363 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa879e41-5e08-4c0a-bfc3-7b2294b4f392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:47.148022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa879e41-5e08-4c0a-bfc3-7b2294b4f392 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:47.148305 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa879e41-5e08-4c0a-bfc3-7b2294b4f392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:47.148583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa879e41-5e08-4c0a-bfc3-7b2294b4f392 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:47.149274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa879e41-5e08-4c0a-bfc3-7b2294b4f392 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:47.151569 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa879e41-5e08-4c0a-bfc3-7b2294b4f392 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:47.188441 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1284/2576] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:47 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:31:47.193463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d498c5-915c-47f1-8c31-0cafe8d3f1d3 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:47.207743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d498c5-915c-47f1-8c31-0cafe8d3f1d3 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:47.211845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:47.280364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e5c36-d09d-4d0c-a772-339cc50d4f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:47.281017 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e5c36-d09d-4d0c-a772-339cc50d4f82 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:47.281881 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e5c36-d09d-4d0c-a772-339cc50d4f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:47.282023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e5c36-d09d-4d0c-a772-339cc50d4f82 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:47.282872 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e5c36-d09d-4d0c-a772-339cc50d4f82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:47.284899 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e5c36-d09d-4d0c-a772-339cc50d4f82 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:47.315795 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1293/2577] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:47 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:47.320361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa879e41-5e08-4c0a-bfc3-7b2294b4f392 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:47.332025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa879e41-5e08-4c0a-bfc3-7b2294b4f392 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:47.336808 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18385-3f8c-4f57-9642-7789f19ca649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:47.337442 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18385-3f8c-4f57-9642-7789f19ca649 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:47.337723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18385-3f8c-4f57-9642-7789f19ca649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:47.338046 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18385-3f8c-4f57-9642-7789f19ca649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:47.338829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb18385-3f8c-4f57-9642-7789f19ca649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:47.341242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb18385-3f8c-4f57-9642-7789f19ca649 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:47.364010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1285/2578] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:47.365312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e5c36-d09d-4d0c-a772-339cc50d4f82 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:47.376443 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e5c36-d09d-4d0c-a772-339cc50d4f82 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:47.382557 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00578ec-e4fb-4e63-bfcb-4dd8d2f58eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:47.383199 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00578ec-e4fb-4e63-bfcb-4dd8d2f58eff admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:47.383476 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00578ec-e4fb-4e63-bfcb-4dd8d2f58eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:47.383766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00578ec-e4fb-4e63-bfcb-4dd8d2f58eff admin admin] PATH_INFO: `/v3/projects/9e9de624f55e4e9fbde35ef26ab16356/users/db80c733c7bd4e8da0ccd756da987077/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:47.403068 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00578ec-e4fb-4e63-bfcb-4dd8d2f58eff admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e9de624f55e4e9fbde35ef26ab16356, user_id=db80c733c7bd4e8da0ccd756da987077, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:47.405878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00578ec-e4fb-4e63-bfcb-4dd8d2f58eff admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:47.423320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1294/2579] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:47 2022] PUT /identity/v3/projects/9e9de624f55e4e9fbde35ef26ab16356/users/db80c733c7bd4e8da0ccd756da987077/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:47.427496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18385-3f8c-4f57-9642-7789f19ca649 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:47.442053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18385-3f8c-4f57-9642-7789f19ca649 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:47.545862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832ae09-521a-454c-b67b-9f950f6833db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:47.546530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832ae09-521a-454c-b67b-9f950f6833db admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:47.546923 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832ae09-521a-454c-b67b-9f950f6833db admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:47.547329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832ae09-521a-454c-b67b-9f950f6833db admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:47.548100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b832ae09-521a-454c-b67b-9f950f6833db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:47.549878 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b832ae09-521a-454c-b67b-9f950f6833db admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:47.563780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1286/2580] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:31:47.566880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00578ec-e4fb-4e63-bfcb-4dd8d2f58eff admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:47.584312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00578ec-e4fb-4e63-bfcb-4dd8d2f58eff admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:47.589856 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe936af2-c2d3-49d8-b92c-8f445ad17a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:47.590667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe936af2-c2d3-49d8-b92c-8f445ad17a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:47.591010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe936af2-c2d3-49d8-b92c-8f445ad17a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:47.591368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe936af2-c2d3-49d8-b92c-8f445ad17a27 admin admin] PATH_INFO: `/v3/projects/9e9de624f55e4e9fbde35ef26ab16356/users/db80c733c7bd4e8da0ccd756da987077/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:47.593549 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe936af2-c2d3-49d8-b92c-8f445ad17a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e9de624f55e4e9fbde35ef26ab16356, user_id=db80c733c7bd4e8da0ccd756da987077, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:47.596639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe936af2-c2d3-49d8-b92c-8f445ad17a27 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:47.622298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1295/2581] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:47 2022] PUT /identity/v3/projects/9e9de624f55e4e9fbde35ef26ab16356/users/db80c733c7bd4e8da0ccd756da987077/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:47.634017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f00ff9-832b-4406-8046-94dfca5f0de4 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:47.634400 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f00ff9-832b-4406-8046-94dfca5f0de4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:47.634755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f00ff9-832b-4406-8046-94dfca5f0de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:47.671536 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-69f00ff9-832b-4406-8046-94dfca5f0de4 None None] MFA Rules not processed for user `db80c733c7bd4e8da0ccd756da987077`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:47.672968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-69f00ff9-832b-4406-8046-94dfca5f0de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:47.751138 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1287/2582] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:47 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:47.763382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe936af2-c2d3-49d8-b92c-8f445ad17a27 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:47.766288 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-fe936af2-c2d3-49d8-b92c-8f445ad17a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:47.838792 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe936af2-c2d3-49d8-b92c-8f445ad17a27 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:47.842585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcba599-cb8f-4720-b013-53b2fa3c10a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:47.843267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcba599-cb8f-4720-b013-53b2fa3c10a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:47.843866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcba599-cb8f-4720-b013-53b2fa3c10a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:47.844233 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcba599-cb8f-4720-b013-53b2fa3c10a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:47.867544 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcba599-cb8f-4720-b013-53b2fa3c10a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:47.871501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcba599-cb8f-4720-b013-53b2fa3c10a4 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:47.906354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1296/2583] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:48.039380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f00ff9-832b-4406-8046-94dfca5f0de4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:48.050645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f00ff9-832b-4406-8046-94dfca5f0de4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:48.056103 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6efbac-d62b-44b1-8bf6-a32f05038261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:48.056846 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6efbac-d62b-44b1-8bf6-a32f05038261 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:48.057257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6efbac-d62b-44b1-8bf6-a32f05038261 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:48.057647 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6efbac-d62b-44b1-8bf6-a32f05038261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:48.138370 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:48.167411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6efbac-d62b-44b1-8bf6-a32f05038261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:48.169328 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6efbac-d62b-44b1-8bf6-a32f05038261 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:48.180763 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1288/2584] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:48 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:31:48.628771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcba599-cb8f-4720-b013-53b2fa3c10a4 service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:48.630398 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6fcba599-cb8f-4720-b013-53b2fa3c10a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:48.698761 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcba599-cb8f-4720-b013-53b2fa3c10a4 service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:48.703859 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573bc65e-54c8-4c7e-87db-e73d9c4853ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:48.704505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bc65e-54c8-4c7e-87db-e73d9c4853ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:48.704783 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bc65e-54c8-4c7e-87db-e73d9c4853ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:48.705054 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bc65e-54c8-4c7e-87db-e73d9c4853ff admin admin] PATH_INFO: `/v3/users/c60ee64ca18e4930a14e3e17bc979ff7` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:48.706285 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573bc65e-54c8-4c7e-87db-e73d9c4853ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=c60ee64ca18e4930a14e3e17bc979ff7)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:48.708266 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573bc65e-54c8-4c7e-87db-e73d9c4853ff admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:48.710680 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:48.758281 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-573bc65e-54c8-4c7e-87db-e73d9c4853ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c60ee64ca18e4930a14e3e17bc979ff7'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:48.767155 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-573bc65e-54c8-4c7e-87db-e73d9c4853ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c60ee64ca18e4930a14e3e17bc979ff7'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:48.767884 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-573bc65e-54c8-4c7e-87db-e73d9c4853ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c60ee64ca18e4930a14e3e17bc979ff7'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:48.775513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-573bc65e-54c8-4c7e-87db-e73d9c4853ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c60ee64ca18e4930a14e3e17bc979ff7'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:48.788003 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-573bc65e-54c8-4c7e-87db-e73d9c4853ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c60ee64ca18e4930a14e3e17bc979ff7'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:48.794054 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1297/2585] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:31:48 2022] DELETE /identity/v3/users/c60ee64ca18e4930a14e3e17bc979ff7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:31:48.981069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6efbac-d62b-44b1-8bf6-a32f05038261 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:49.007372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6efbac-d62b-44b1-8bf6-a32f05038261 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:49.071681 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c59f3-eaab-4881-b61b-68cc0492247a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:49.072355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c59f3-eaab-4881-b61b-68cc0492247a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:49.072637 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c59f3-eaab-4881-b61b-68cc0492247a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:49.072937 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c59f3-eaab-4881-b61b-68cc0492247a admin admin] PATH_INFO: `/v3/projects/5400c4df102f43398a278247c4431ef6` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:49.074195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c59f3-eaab-4881-b61b-68cc0492247a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5400c4df102f43398a278247c4431ef6)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:49.076115 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c59f3-eaab-4881-b61b-68cc0492247a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:49.123795 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-231c59f3-eaab-4881-b61b-68cc0492247a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5400c4df102f43398a278247c4431ef6', 'request_id': 'req-231c59f3-eaab-4881-b61b-68cc0492247a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:49.129563 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-231c59f3-eaab-4881-b61b-68cc0492247a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5400c4df102f43398a278247c4431ef6', 'request_id': 'req-231c59f3-eaab-4881-b61b-68cc0492247a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:49.137831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-231c59f3-eaab-4881-b61b-68cc0492247a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5400c4df102f43398a278247c4431ef6', 'request_id': 'req-231c59f3-eaab-4881-b61b-68cc0492247a'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:49.150240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-231c59f3-eaab-4881-b61b-68cc0492247a admin admin] The token cache is being invalidate because project 5400c4df102f43398a278247c4431ef6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:31:49.150522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-231c59f3-eaab-4881-b61b-68cc0492247a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:49.151230 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-231c59f3-eaab-4881-b61b-68cc0492247a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:49.152849 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1289/2586] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:31:48 2022] DELETE /identity/v3/projects/5400c4df102f43398a278247c4431ef6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:49.158420 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3b9d3-cdc4-4074-b687-c7790d484468 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:49.158584 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3b9d3-cdc4-4074-b687-c7790d484468 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:49.158903 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3b9d3-cdc4-4074-b687-c7790d484468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:49.179411 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-a0b3b9d3-cdc4-4074-b687-c7790d484468 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:49.180632 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-a0b3b9d3-cdc4-4074-b687-c7790d484468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:49.242595 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1298/2587] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:49.250586 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af275b-6980-4fe1-8558-6befe9d99d9d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:49.251105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af275b-6980-4fe1-8558-6befe9d99d9d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:49.251105 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af275b-6980-4fe1-8558-6befe9d99d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:49.271710 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-76af275b-6980-4fe1-8558-6befe9d99d9d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:49.272931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-76af275b-6980-4fe1-8558-6befe9d99d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:49.281404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1290/2588] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:49.286133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3b9d3-cdc4-4074-b687-c7790d484468 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:49.296483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3b9d3-cdc4-4074-b687-c7790d484468 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:49.302581 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859dbcb7-4ed3-40ed-bee9-4c835031a4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:49.303212 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dbcb7-4ed3-40ed-bee9-4c835031a4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:49.303490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dbcb7-4ed3-40ed-bee9-4c835031a4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:49.303758 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859dbcb7-4ed3-40ed-bee9-4c835031a4a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:49.304467 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859dbcb7-4ed3-40ed-bee9-4c835031a4a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:31:49.304772 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859dbcb7-4ed3-40ed-bee9-4c835031a4a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:49.306425 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859dbcb7-4ed3-40ed-bee9-4c835031a4a4 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:49.319166 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1299/2589] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:31:49.323606 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af275b-6980-4fe1-8558-6befe9d99d9d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:49.333816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af275b-6980-4fe1-8558-6befe9d99d9d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:49.339049 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5f33a-698d-4953-9a77-ac2fe2cb511a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:49.339674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5f33a-698d-4953-9a77-ac2fe2cb511a admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:49.339956 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5f33a-698d-4953-9a77-ac2fe2cb511a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:49.340226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5f33a-698d-4953-9a77-ac2fe2cb511a admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:49.340897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5f33a-698d-4953-9a77-ac2fe2cb511a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:49.342878 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5f33a-698d-4953-9a77-ac2fe2cb511a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:49.374169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1291/2590] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:49 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:49.378450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859dbcb7-4ed3-40ed-bee9-4c835031a4a4 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:49.389456 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859dbcb7-4ed3-40ed-bee9-4c835031a4a4 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:49.450115 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6b326-598a-4ae7-b211-67efdea3709a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:49.450845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6b326-598a-4ae7-b211-67efdea3709a admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:49.451250 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6b326-598a-4ae7-b211-67efdea3709a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:49.451477 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6b326-598a-4ae7-b211-67efdea3709a admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:49.452110 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c6b326-598a-4ae7-b211-67efdea3709a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:49.455380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c6b326-598a-4ae7-b211-67efdea3709a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:49.475491 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:49.485928 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1300/2591] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:49 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:31:49.489301 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5f33a-698d-4953-9a77-ac2fe2cb511a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:49.505698 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5f33a-698d-4953-9a77-ac2fe2cb511a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:49.512278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d1b97-7a6a-4038-aea7-0781390facf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:49.512912 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d1b97-7a6a-4038-aea7-0781390facf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:49.513204 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d1b97-7a6a-4038-aea7-0781390facf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:49.513577 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d1b97-7a6a-4038-aea7-0781390facf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:49.514383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d1b97-7a6a-4038-aea7-0781390facf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:49.515959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d1b97-7a6a-4038-aea7-0781390facf7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:49.529943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1292/2592] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:49.538595 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6b326-598a-4ae7-b211-67efdea3709a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:49.553586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6b326-598a-4ae7-b211-67efdea3709a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:49.563172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec7bea-b8a6-463e-a8cd-69e6f257d37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:49.564377 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec7bea-b8a6-463e-a8cd-69e6f257d37c admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:49.565025 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec7bea-b8a6-463e-a8cd-69e6f257d37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:49.565494 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec7bea-b8a6-463e-a8cd-69e6f257d37c admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:49.567557 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec7bea-b8a6-463e-a8cd-69e6f257d37c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:49.570709 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec7bea-b8a6-463e-a8cd-69e6f257d37c admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:49.598613 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1301/2593] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:49.601258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d1b97-7a6a-4038-aea7-0781390facf7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:49.620621 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d1b97-7a6a-4038-aea7-0781390facf7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:49.629068 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1d1e6-d4d6-44bb-8bee-0df469b8f579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:49.630083 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1d1e6-d4d6-44bb-8bee-0df469b8f579 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:49.630663 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1d1e6-d4d6-44bb-8bee-0df469b8f579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:49.631154 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b1d1e6-d4d6-44bb-8bee-0df469b8f579 admin admin] PATH_INFO: `/v3/projects/9d77dd15ce9b4352ad2a9c2c3605bc18/users/20a34732bd8c49b19afee415ee58a1ed/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:49.659043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1d1e6-d4d6-44bb-8bee-0df469b8f579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d77dd15ce9b4352ad2a9c2c3605bc18, user_id=20a34732bd8c49b19afee415ee58a1ed, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:49.664536 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b1d1e6-d4d6-44bb-8bee-0df469b8f579 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:49.699963 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1293/2594] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:49 2022] PUT /identity/v3/projects/9d77dd15ce9b4352ad2a9c2c3605bc18/users/20a34732bd8c49b19afee415ee58a1ed/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:49.711770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99731b-7f4b-4f07-9866-0539b9a2c3f8 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:49.711770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99731b-7f4b-4f07-9866-0539b9a2c3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:49.712604 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99731b-7f4b-4f07-9866-0539b9a2c3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:49.749483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-0b99731b-7f4b-4f07-9866-0539b9a2c3f8 None None] MFA Rules not processed for user `20a34732bd8c49b19afee415ee58a1ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:49.750691 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-0b99731b-7f4b-4f07-9866-0539b9a2c3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:49.856703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1302/2595] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:49.874800 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1d1e6-d4d6-44bb-8bee-0df469b8f579 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:49.877490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-57b1d1e6-d4d6-44bb-8bee-0df469b8f579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:49.901943 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:50.057050 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99731b-7f4b-4f07-9866-0539b9a2c3f8 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:50.058769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b1d1e6-d4d6-44bb-8bee-0df469b8f579 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:50.060314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-0b99731b-7f4b-4f07-9866-0539b9a2c3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:50.066889 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3d0fe-edc0-4b44-8f4d-2d36f3c88a54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:50.067530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3d0fe-edc0-4b44-8f4d-2d36f3c88a54 service nova] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:50.068082 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3d0fe-edc0-4b44-8f4d-2d36f3c88a54 service nova] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:50.068372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3d0fe-edc0-4b44-8f4d-2d36f3c88a54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:50.082118 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3d0fe-edc0-4b44-8f4d-2d36f3c88a54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:50.085986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3d0fe-edc0-4b44-8f4d-2d36f3c88a54 service nova] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:50.095422 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1294/2596] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:50.137704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99731b-7f4b-4f07-9866-0539b9a2c3f8 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:50.141627 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e4b45-f723-42da-a85f-95f129f6000c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:50.142289 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e4b45-f723-42da-a85f-95f129f6000c service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:50.142565 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e4b45-f723-42da-a85f-95f129f6000c service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:50.142997 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e4b45-f723-42da-a85f-95f129f6000c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:50.144785 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5f9e4b45-f723-42da-a85f-95f129f6000c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:50.230391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e4b45-f723-42da-a85f-95f129f6000c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:50.231986 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e4b45-f723-42da-a85f-95f129f6000c service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:50.242834 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1303/2597] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:31:50.324729 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3d0fe-edc0-4b44-8f4d-2d36f3c88a54 service nova] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:50.398809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3d0fe-edc0-4b44-8f4d-2d36f3c88a54 service nova] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:50.404056 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac645fa-703d-42bd-9c5a-151895e57b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:50.404691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac645fa-703d-42bd-9c5a-151895e57b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:50.404972 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac645fa-703d-42bd-9c5a-151895e57b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:50.405214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac645fa-703d-42bd-9c5a-151895e57b4e admin admin] PATH_INFO: `/v3/users/20a34732bd8c49b19afee415ee58a1ed` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:50.406722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac645fa-703d-42bd-9c5a-151895e57b4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=20a34732bd8c49b19afee415ee58a1ed)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:50.409189 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac645fa-703d-42bd-9c5a-151895e57b4e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:50.466748 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-dac645fa-703d-42bd-9c5a-151895e57b4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20a34732bd8c49b19afee415ee58a1ed'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:50.467464 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-dac645fa-703d-42bd-9c5a-151895e57b4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20a34732bd8c49b19afee415ee58a1ed'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:50.481259 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-dac645fa-703d-42bd-9c5a-151895e57b4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20a34732bd8c49b19afee415ee58a1ed'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:50.492780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-dac645fa-703d-42bd-9c5a-151895e57b4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20a34732bd8c49b19afee415ee58a1ed'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:50.495537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:50.502959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-dac645fa-703d-42bd-9c5a-151895e57b4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20a34732bd8c49b19afee415ee58a1ed'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:50.510628 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1295/2598] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:31:50 2022] DELETE /identity/v3/users/20a34732bd8c49b19afee415ee58a1ed => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:31:50.515469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b9215-2442-4ceb-9982-27f9106e406a None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:50.515911 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b9215-2442-4ceb-9982-27f9106e406a None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:50.516286 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b9215-2442-4ceb-9982-27f9106e406a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:50.536474 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-687b9215-2442-4ceb-9982-27f9106e406a None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:50.539311 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-687b9215-2442-4ceb-9982-27f9106e406a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:50.562817 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:50.607891 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1304/2599] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:50.619877 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac645fa-703d-42bd-9c5a-151895e57b4e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:50.621453 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-dac645fa-703d-42bd-9c5a-151895e57b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:50.690412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac645fa-703d-42bd-9c5a-151895e57b4e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:50.694676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de5b4a-9fd3-42a7-b1ea-a94ac304d8a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:50.695321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de5b4a-9fd3-42a7-b1ea-a94ac304d8a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:50.695600 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de5b4a-9fd3-42a7-b1ea-a94ac304d8a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:50.695874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de5b4a-9fd3-42a7-b1ea-a94ac304d8a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:50.706532 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de5b4a-9fd3-42a7-b1ea-a94ac304d8a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:50.707996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de5b4a-9fd3-42a7-b1ea-a94ac304d8a1 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:50.722294 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1296/2600] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:51.356971 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b9215-2442-4ceb-9982-27f9106e406a None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:51.359349 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-687b9215-2442-4ceb-9982-27f9106e406a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:51.389552 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b9215-2442-4ceb-9982-27f9106e406a None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:51.396433 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a6e27-8df4-4123-b0d2-ab52dfec0ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:51.397310 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a6e27-8df4-4123-b0d2-ab52dfec0ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:51.397508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a6e27-8df4-4123-b0d2-ab52dfec0ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:51.397895 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a6e27-8df4-4123-b0d2-ab52dfec0ddf admin admin] PATH_INFO: `/v3/projects/9d77dd15ce9b4352ad2a9c2c3605bc18` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:51.399165 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a6e27-8df4-4123-b0d2-ab52dfec0ddf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d77dd15ce9b4352ad2a9c2c3605bc18)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:51.401173 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a6e27-8df4-4123-b0d2-ab52dfec0ddf admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:51.459045 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0e3a6e27-8df4-4123-b0d2-ab52dfec0ddf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d77dd15ce9b4352ad2a9c2c3605bc18', 'request_id': 'req-0e3a6e27-8df4-4123-b0d2-ab52dfec0ddf'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:51.467933 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0e3a6e27-8df4-4123-b0d2-ab52dfec0ddf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d77dd15ce9b4352ad2a9c2c3605bc18', 'request_id': 'req-0e3a6e27-8df4-4123-b0d2-ab52dfec0ddf'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:51.476186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0e3a6e27-8df4-4123-b0d2-ab52dfec0ddf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d77dd15ce9b4352ad2a9c2c3605bc18', 'request_id': 'req-0e3a6e27-8df4-4123-b0d2-ab52dfec0ddf'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:51.490282 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0e3a6e27-8df4-4123-b0d2-ab52dfec0ddf admin admin] The token cache is being invalidate because project 9d77dd15ce9b4352ad2a9c2c3605bc18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:31:51.490667 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0e3a6e27-8df4-4123-b0d2-ab52dfec0ddf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:51.492939 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0e3a6e27-8df4-4123-b0d2-ab52dfec0ddf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:51.494688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1305/2601] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:31:51 2022] DELETE /identity/v3/projects/9d77dd15ce9b4352ad2a9c2c3605bc18 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:51.499485 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b6042-d679-4022-b7e2-f53eca64a098 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:51.499992 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b6042-d679-4022-b7e2-f53eca64a098 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:51.500099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b6042-d679-4022-b7e2-f53eca64a098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:51.523800 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-620b6042-d679-4022-b7e2-f53eca64a098 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:51.525567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-620b6042-d679-4022-b7e2-f53eca64a098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:51.612382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1297/2602] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:31:51.618275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bd75c-291d-45a0-a097-ab198d983c1e None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:51.618865 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bd75c-291d-45a0-a097-ab198d983c1e None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:51.619210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bd75c-291d-45a0-a097-ab198d983c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:51.644931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-c82bd75c-291d-45a0-a097-ab198d983c1e None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:51.646529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c82bd75c-291d-45a0-a097-ab198d983c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:51.655626 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1306/2603] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:31:51.659716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b6042-d679-4022-b7e2-f53eca64a098 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:51.673583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b6042-d679-4022-b7e2-f53eca64a098 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:51.678495 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760e071-b653-4ad5-bbbb-f440cac3491e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:51.679716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760e071-b653-4ad5-bbbb-f440cac3491e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:51.679998 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760e071-b653-4ad5-bbbb-f440cac3491e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:51.680299 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760e071-b653-4ad5-bbbb-f440cac3491e admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:51.681078 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760e071-b653-4ad5-bbbb-f440cac3491e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:31:51.681383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760e071-b653-4ad5-bbbb-f440cac3491e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:51.683066 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760e071-b653-4ad5-bbbb-f440cac3491e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:51.697183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1298/2604] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:31:51.701997 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82bd75c-291d-45a0-a097-ab198d983c1e None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:51.722041 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82bd75c-291d-45a0-a097-ab198d983c1e None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:51.727635 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7d0e0-edfa-45ed-afe7-a04ce01a28df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:51.728265 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7d0e0-edfa-45ed-afe7-a04ce01a28df admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:51.728543 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7d0e0-edfa-45ed-afe7-a04ce01a28df admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:51.728817 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7d0e0-edfa-45ed-afe7-a04ce01a28df admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:51.729535 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7d0e0-edfa-45ed-afe7-a04ce01a28df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:51.731852 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb7d0e0-edfa-45ed-afe7-a04ce01a28df admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:51.768413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1307/2605] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:51 2022] POST /identity/v3/projects => generated 367 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:51.772233 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760e071-b653-4ad5-bbbb-f440cac3491e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:51.783212 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760e071-b653-4ad5-bbbb-f440cac3491e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:51.843252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c85ff8-bb54-49f4-bf42-fa073cc3f322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:51.843869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c85ff8-bb54-49f4-bf42-fa073cc3f322 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:51.844148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c85ff8-bb54-49f4-bf42-fa073cc3f322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:51.844570 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c85ff8-bb54-49f4-bf42-fa073cc3f322 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:51.845240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c85ff8-bb54-49f4-bf42-fa073cc3f322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:51.847420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c85ff8-bb54-49f4-bf42-fa073cc3f322 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:51.873705 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1299/2606] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:51 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:51.878093 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7d0e0-edfa-45ed-afe7-a04ce01a28df admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:51.891174 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7d0e0-edfa-45ed-afe7-a04ce01a28df admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:51.897346 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40116494-1cfb-4371-b363-e2606817d4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:51.898012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40116494-1cfb-4371-b363-e2606817d4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:51.898261 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40116494-1cfb-4371-b363-e2606817d4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:51.898532 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40116494-1cfb-4371-b363-e2606817d4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:51.899234 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40116494-1cfb-4371-b363-e2606817d4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:51.900884 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40116494-1cfb-4371-b363-e2606817d4d9 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:51.912443 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1308/2607] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:51.918054 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c85ff8-bb54-49f4-bf42-fa073cc3f322 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:51.930097 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c85ff8-bb54-49f4-bf42-fa073cc3f322 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:51.935823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54920c02-05c8-403f-98ec-99e82580b1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:51.936700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54920c02-05c8-403f-98ec-99e82580b1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:51.936982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54920c02-05c8-403f-98ec-99e82580b1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:51.937258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54920c02-05c8-403f-98ec-99e82580b1c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:51.938006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54920c02-05c8-403f-98ec-99e82580b1c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:51.939684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54920c02-05c8-403f-98ec-99e82580b1c4 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:51.945754 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:51.952472 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1300/2608] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:51.957754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40116494-1cfb-4371-b363-e2606817d4d9 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:51.968533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40116494-1cfb-4371-b363-e2606817d4d9 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:51.979349 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0087deb-f64b-4ffb-926b-a19f8740e48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:51.980224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0087deb-f64b-4ffb-926b-a19f8740e48a admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:51.980594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0087deb-f64b-4ffb-926b-a19f8740e48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:51.980997 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0087deb-f64b-4ffb-926b-a19f8740e48a admin admin] PATH_INFO: `/v3/projects/8447f777e85c437cbb4a9d7212f39a10/users/c4582c7b365a4e928001b033a5702341/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:51.999190 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0087deb-f64b-4ffb-926b-a19f8740e48a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8447f777e85c437cbb4a9d7212f39a10, user_id=c4582c7b365a4e928001b033a5702341, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:52.001350 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0087deb-f64b-4ffb-926b-a19f8740e48a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:52.016278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1309/2609] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:51 2022] PUT /identity/v3/projects/8447f777e85c437cbb4a9d7212f39a10/users/c4582c7b365a4e928001b033a5702341/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:52.026959 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50f2a7-1a21-4f09-a438-8bc3fab36ccb None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:52.027325 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50f2a7-1a21-4f09-a438-8bc3fab36ccb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:52.027646 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50f2a7-1a21-4f09-a438-8bc3fab36ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:52.060833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-6c50f2a7-1a21-4f09-a438-8bc3fab36ccb None None] MFA Rules not processed for user `c4582c7b365a4e928001b033a5702341`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:52.062117 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-6c50f2a7-1a21-4f09-a438-8bc3fab36ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:52.193850 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1301/2610] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:52.196234 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0087deb-f64b-4ffb-926b-a19f8740e48a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:52.249838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:52.267240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0087deb-f64b-4ffb-926b-a19f8740e48a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:52.272033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03e058-11ea-4059-b452-da763c0138d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:52.272675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03e058-11ea-4059-b452-da763c0138d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:52.272948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03e058-11ea-4059-b452-da763c0138d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:52.273225 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03e058-11ea-4059-b452-da763c0138d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:52.273922 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d03e058-11ea-4059-b452-da763c0138d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:52.275838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d03e058-11ea-4059-b452-da763c0138d0 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:52.312167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1310/2611] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:52 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:52.313489 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50f2a7-1a21-4f09-a438-8bc3fab36ccb None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:52.324822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50f2a7-1a21-4f09-a438-8bc3fab36ccb None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:52.392516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ad0ba-052a-48ad-8d46-938a66c9a95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:52.393199 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ad0ba-052a-48ad-8d46-938a66c9a95d admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:52.393456 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ad0ba-052a-48ad-8d46-938a66c9a95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:52.393745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ad0ba-052a-48ad-8d46-938a66c9a95d admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:52.394493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ad0ba-052a-48ad-8d46-938a66c9a95d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:52.396695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ad0ba-052a-48ad-8d46-938a66c9a95d admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:52.431012 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1302/2612] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:52 2022] POST /identity/v3/users => generated 290 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:52.433563 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03e058-11ea-4059-b452-da763c0138d0 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:52.447827 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03e058-11ea-4059-b452-da763c0138d0 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:52.456413 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b423e87-2470-44ba-8d90-676771470671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:52.457038 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b423e87-2470-44ba-8d90-676771470671 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:52.457266 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b423e87-2470-44ba-8d90-676771470671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:52.457525 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b423e87-2470-44ba-8d90-676771470671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:52.458271 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b423e87-2470-44ba-8d90-676771470671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:52.460159 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b423e87-2470-44ba-8d90-676771470671 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:52.471701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1311/2613] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:52.476490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ad0ba-052a-48ad-8d46-938a66c9a95d admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:52.487630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ad0ba-052a-48ad-8d46-938a66c9a95d admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:52.493995 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bca7d2-88c9-42e4-aafa-a9fee7ed060b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:52.495053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bca7d2-88c9-42e4-aafa-a9fee7ed060b admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:52.495575 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bca7d2-88c9-42e4-aafa-a9fee7ed060b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:52.496010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bca7d2-88c9-42e4-aafa-a9fee7ed060b admin admin] PATH_INFO: `/v3/projects/acb7533eae2c4e188ce1a4907e107089/users/fcb0afa1562b4633a31c31c7088e9f34/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:52.513178 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bca7d2-88c9-42e4-aafa-a9fee7ed060b admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb7533eae2c4e188ce1a4907e107089, user_id=fcb0afa1562b4633a31c31c7088e9f34, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:52.516127 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bca7d2-88c9-42e4-aafa-a9fee7ed060b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:52.530988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1303/2614] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:52 2022] PUT /identity/v3/projects/acb7533eae2c4e188ce1a4907e107089/users/fcb0afa1562b4633a31c31c7088e9f34/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:52.535417 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b423e87-2470-44ba-8d90-676771470671 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:52.550224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b423e87-2470-44ba-8d90-676771470671 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:52.612869 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eafe3b8-15a1-4dd2-b582-d0bab0f12570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:52.613516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafe3b8-15a1-4dd2-b582-d0bab0f12570 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:52.613799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafe3b8-15a1-4dd2-b582-d0bab0f12570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:52.614094 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafe3b8-15a1-4dd2-b582-d0bab0f12570 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:52.614895 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eafe3b8-15a1-4dd2-b582-d0bab0f12570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:52.616592 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eafe3b8-15a1-4dd2-b582-d0bab0f12570 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:52.627478 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1312/2615] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:52.631703 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bca7d2-88c9-42e4-aafa-a9fee7ed060b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:52.642507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bca7d2-88c9-42e4-aafa-a9fee7ed060b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:52.648242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e5961-d8b0-4100-9bc2-ae39ea09748f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:52.649009 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e5961-d8b0-4100-9bc2-ae39ea09748f admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:52.649356 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e5961-d8b0-4100-9bc2-ae39ea09748f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:52.649694 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e5961-d8b0-4100-9bc2-ae39ea09748f admin admin] PATH_INFO: `/v3/projects/acb7533eae2c4e188ce1a4907e107089/users/fcb0afa1562b4633a31c31c7088e9f34/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:52.651874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e5961-d8b0-4100-9bc2-ae39ea09748f admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb7533eae2c4e188ce1a4907e107089, user_id=fcb0afa1562b4633a31c31c7088e9f34, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:52.654667 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e5961-d8b0-4100-9bc2-ae39ea09748f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:52.667498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1304/2616] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:52 2022] PUT /identity/v3/projects/acb7533eae2c4e188ce1a4907e107089/users/fcb0afa1562b4633a31c31c7088e9f34/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:52.678264 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff994b5f-eb6f-4141-b266-2c95a149cb91 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:52.678527 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff994b5f-eb6f-4141-b266-2c95a149cb91 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:52.678841 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff994b5f-eb6f-4141-b266-2c95a149cb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:52.709010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-ff994b5f-eb6f-4141-b266-2c95a149cb91 None None] MFA Rules not processed for user `fcb0afa1562b4633a31c31c7088e9f34`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:52.710305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-ff994b5f-eb6f-4141-b266-2c95a149cb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:52.782095 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1313/2617] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:52 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:52.793461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e5961-d8b0-4100-9bc2-ae39ea09748f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:52.795263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-726e5961-d8b0-4100-9bc2-ae39ea09748f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:52.856726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:52.880848 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e5961-d8b0-4100-9bc2-ae39ea09748f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:52.884955 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8a0cc-f671-45b8-8c49-4841f59787bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:52.885609 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8a0cc-f671-45b8-8c49-4841f59787bc service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:52.885921 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8a0cc-f671-45b8-8c49-4841f59787bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:52.886192 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8a0cc-f671-45b8-8c49-4841f59787bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:52.896411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8a0cc-f671-45b8-8c49-4841f59787bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:52.898546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8a0cc-f671-45b8-8c49-4841f59787bc service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:52.910115 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1305/2618] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:52 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:54.462668 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff994b5f-eb6f-4141-b266-2c95a149cb91 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:54.511374 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:54.536414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff994b5f-eb6f-4141-b266-2c95a149cb91 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:54.541881 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f035a29-7e4a-4b35-b9c9-4d2986536a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:54.542606 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f035a29-7e4a-4b35-b9c9-4d2986536a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:54.542944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f035a29-7e4a-4b35-b9c9-4d2986536a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:54.543331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f035a29-7e4a-4b35-b9c9-4d2986536a6e admin admin] PATH_INFO: `/v3/users/c4582c7b365a4e928001b033a5702341` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:54.544679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f035a29-7e4a-4b35-b9c9-4d2986536a6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4582c7b365a4e928001b033a5702341)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:54.547044 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f035a29-7e4a-4b35-b9c9-4d2986536a6e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:54.605832 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6f035a29-7e4a-4b35-b9c9-4d2986536a6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4582c7b365a4e928001b033a5702341'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:54.614833 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6f035a29-7e4a-4b35-b9c9-4d2986536a6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4582c7b365a4e928001b033a5702341'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:54.615609 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6f035a29-7e4a-4b35-b9c9-4d2986536a6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4582c7b365a4e928001b033a5702341'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:54.623140 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6f035a29-7e4a-4b35-b9c9-4d2986536a6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4582c7b365a4e928001b033a5702341'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:54.636611 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-6f035a29-7e4a-4b35-b9c9-4d2986536a6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4582c7b365a4e928001b033a5702341'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:54.650322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1314/2619] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:31:54 2022] DELETE /identity/v3/users/c4582c7b365a4e928001b033a5702341 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:54.654746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f278e-484a-4b7e-a3dc-16a305dfa864 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:54.655279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f278e-484a-4b7e-a3dc-16a305dfa864 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:54.655494 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f278e-484a-4b7e-a3dc-16a305dfa864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:54.675713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-1a8f278e-484a-4b7e-a3dc-16a305dfa864 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:54.677005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1a8f278e-484a-4b7e-a3dc-16a305dfa864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:54.743387 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1306/2620] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:54.753397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f035a29-7e4a-4b35-b9c9-4d2986536a6e admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:54.754863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6f035a29-7e4a-4b35-b9c9-4d2986536a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:54.816184 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f035a29-7e4a-4b35-b9c9-4d2986536a6e admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:54.820570 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9b81c-9715-46d0-82c3-c8f31ae8ff38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:54.821182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9b81c-9715-46d0-82c3-c8f31ae8ff38 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:54.821490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9b81c-9715-46d0-82c3-c8f31ae8ff38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:54.821759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9b81c-9715-46d0-82c3-c8f31ae8ff38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:54.843167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9b81c-9715-46d0-82c3-c8f31ae8ff38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:54.849120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9b81c-9715-46d0-82c3-c8f31ae8ff38 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:54.860365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1315/2621] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:31:55.172065 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f278e-484a-4b7e-a3dc-16a305dfa864 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:55.173718 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1a8f278e-484a-4b7e-a3dc-16a305dfa864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:55.206289 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f278e-484a-4b7e-a3dc-16a305dfa864 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:55.211391 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f266a4-936f-438e-825a-66ea9a26c336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:55.212064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f266a4-936f-438e-825a-66ea9a26c336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:55.212346 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f266a4-936f-438e-825a-66ea9a26c336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:55.212660 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f266a4-936f-438e-825a-66ea9a26c336 admin admin] PATH_INFO: `/v3/projects/8447f777e85c437cbb4a9d7212f39a10` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:55.213885 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f266a4-936f-438e-825a-66ea9a26c336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8447f777e85c437cbb4a9d7212f39a10)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:55.215850 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f266a4-936f-438e-825a-66ea9a26c336 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:55.280443 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a2f266a4-936f-438e-825a-66ea9a26c336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8447f777e85c437cbb4a9d7212f39a10', 'request_id': 'req-a2f266a4-936f-438e-825a-66ea9a26c336'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:55.286774 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a2f266a4-936f-438e-825a-66ea9a26c336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8447f777e85c437cbb4a9d7212f39a10', 'request_id': 'req-a2f266a4-936f-438e-825a-66ea9a26c336'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:55.295818 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a2f266a4-936f-438e-825a-66ea9a26c336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8447f777e85c437cbb4a9d7212f39a10', 'request_id': 'req-a2f266a4-936f-438e-825a-66ea9a26c336'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:55.308265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a2f266a4-936f-438e-825a-66ea9a26c336 admin admin] The token cache is being invalidate because project 8447f777e85c437cbb4a9d7212f39a10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:31:55.308550 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a2f266a4-936f-438e-825a-66ea9a26c336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:55.309252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-a2f266a4-936f-438e-825a-66ea9a26c336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:55.310808 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1307/2622] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:31:55 2022] DELETE /identity/v3/projects/8447f777e85c437cbb4a9d7212f39a10 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:55.315221 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9b81c-9715-46d0-82c3-c8f31ae8ff38 service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:55.325490 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9b81c-9715-46d0-82c3-c8f31ae8ff38 service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:55.384131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90d265-ce64-4df6-ab4c-4dccd7e0997b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:55.384685 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90d265-ce64-4df6-ab4c-4dccd7e0997b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:55.384951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90d265-ce64-4df6-ab4c-4dccd7e0997b admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:55.385235 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90d265-ce64-4df6-ab4c-4dccd7e0997b admin admin] PATH_INFO: `/v3/users/fcb0afa1562b4633a31c31c7088e9f34` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:55.386424 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90d265-ce64-4df6-ab4c-4dccd7e0997b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcb0afa1562b4633a31c31c7088e9f34)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:55.388364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90d265-ce64-4df6-ab4c-4dccd7e0997b admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:55.393368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:55.438031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-5d90d265-ce64-4df6-ab4c-4dccd7e0997b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcb0afa1562b4633a31c31c7088e9f34'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:55.446135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-5d90d265-ce64-4df6-ab4c-4dccd7e0997b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcb0afa1562b4633a31c31c7088e9f34'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:55.446770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-5d90d265-ce64-4df6-ab4c-4dccd7e0997b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcb0afa1562b4633a31c31c7088e9f34'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:55.455466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-5d90d265-ce64-4df6-ab4c-4dccd7e0997b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcb0afa1562b4633a31c31c7088e9f34'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:55.468343 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-5d90d265-ce64-4df6-ab4c-4dccd7e0997b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcb0afa1562b4633a31c31c7088e9f34'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:55.474798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1316/2623] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:31:55 2022] DELETE /identity/v3/users/fcb0afa1562b4633a31c31c7088e9f34 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:55.610871 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f266a4-936f-438e-825a-66ea9a26c336 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:55.612498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a2f266a4-936f-438e-825a-66ea9a26c336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:55.712296 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:55.742046 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f266a4-936f-438e-825a-66ea9a26c336 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:55.773297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90d265-ce64-4df6-ab4c-4dccd7e0997b admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:55.774804 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5d90d265-ce64-4df6-ab4c-4dccd7e0997b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:55.823108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab23f6-f91f-486c-8a58-17c3dbe4b787 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '098d8d7e35804ccc931ddfba07f688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:55.823905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab23f6-f91f-486c-8a58-17c3dbe4b787 service placement] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:55.824183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab23f6-f91f-486c-8a58-17c3dbe4b787 service placement] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:55.824451 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab23f6-f91f-486c-8a58-17c3dbe4b787 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:55.826399 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-09ab23f6-f91f-486c-8a58-17c3dbe4b787 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:55.839936 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab23f6-f91f-486c-8a58-17c3dbe4b787 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:55.841769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab23f6-f91f-486c-8a58-17c3dbe4b787 service placement] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:55.852315 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1308/2624] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:55.870863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90d265-ce64-4df6-ab4c-4dccd7e0997b admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:55.876336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c8be6-5f49-4c3e-a4fa-132779059382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:55.877112 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c8be6-5f49-4c3e-a4fa-132779059382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:55.877397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c8be6-5f49-4c3e-a4fa-132779059382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:55.877741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c8be6-5f49-4c3e-a4fa-132779059382 admin admin] PATH_INFO: `/v3/projects/acb7533eae2c4e188ce1a4907e107089` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:55.879201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c8be6-5f49-4c3e-a4fa-132779059382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb7533eae2c4e188ce1a4907e107089)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:55.881234 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c8be6-5f49-4c3e-a4fa-132779059382 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:55.939414 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8a1c8be6-5f49-4c3e-a4fa-132779059382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb7533eae2c4e188ce1a4907e107089', 'request_id': 'req-8a1c8be6-5f49-4c3e-a4fa-132779059382'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:55.949535 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8a1c8be6-5f49-4c3e-a4fa-132779059382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb7533eae2c4e188ce1a4907e107089', 'request_id': 'req-8a1c8be6-5f49-4c3e-a4fa-132779059382'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:55.957091 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8a1c8be6-5f49-4c3e-a4fa-132779059382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb7533eae2c4e188ce1a4907e107089', 'request_id': 'req-8a1c8be6-5f49-4c3e-a4fa-132779059382'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:55.968439 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8a1c8be6-5f49-4c3e-a4fa-132779059382 admin admin] The token cache is being invalidate because project acb7533eae2c4e188ce1a4907e107089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:31:55.968730 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8a1c8be6-5f49-4c3e-a4fa-132779059382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:55.969436 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8a1c8be6-5f49-4c3e-a4fa-132779059382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:55.971020 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1317/2625] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:31:55 2022] DELETE /identity/v3/projects/acb7533eae2c4e188ce1a4907e107089 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:31:55.978505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd80544-1607-430e-83f7-08dc6832fcaf None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:55.978909 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd80544-1607-430e-83f7-08dc6832fcaf None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:55.979382 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd80544-1607-430e-83f7-08dc6832fcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.011479 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-afd80544-1607-430e-83f7-08dc6832fcaf None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:56.012707 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-afd80544-1607-430e-83f7-08dc6832fcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:56.085852 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1309/2626] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:56.095209 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d09dfb-fd3f-42ed-ae9d-4943fe02d282 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:56.095497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d09dfb-fd3f-42ed-ae9d-4943fe02d282 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:56.095770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d09dfb-fd3f-42ed-ae9d-4943fe02d282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.118108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-d1d09dfb-fd3f-42ed-ae9d-4943fe02d282 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:56.119205 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d1d09dfb-fd3f-42ed-ae9d-4943fe02d282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:56.128390 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1318/2627] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:56.132708 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd80544-1607-430e-83f7-08dc6832fcaf None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:56.143167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd80544-1607-430e-83f7-08dc6832fcaf None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:56.148236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d3c6c-c6be-4cab-96d7-4b60e3d90214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:56.148857 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d3c6c-c6be-4cab-96d7-4b60e3d90214 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:56.149136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d3c6c-c6be-4cab-96d7-4b60e3d90214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:56.149413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d3c6c-c6be-4cab-96d7-4b60e3d90214 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.150148 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d3c6c-c6be-4cab-96d7-4b60e3d90214 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:31:56.150447 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d3c6c-c6be-4cab-96d7-4b60e3d90214 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:56.152066 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d3c6c-c6be-4cab-96d7-4b60e3d90214 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:56.163741 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1310/2628] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:31:56.168235 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d09dfb-fd3f-42ed-ae9d-4943fe02d282 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:56.178045 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d09dfb-fd3f-42ed-ae9d-4943fe02d282 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:56.183237 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21fdce-77b4-4043-a52e-dddaa734b273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:56.183855 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21fdce-77b4-4043-a52e-dddaa734b273 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:56.184127 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21fdce-77b4-4043-a52e-dddaa734b273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:56.184399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21fdce-77b4-4043-a52e-dddaa734b273 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.185077 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21fdce-77b4-4043-a52e-dddaa734b273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:56.187026 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21fdce-77b4-4043-a52e-dddaa734b273 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:56.192073 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:56.219242 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1319/2629] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:56 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:31:56.223991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d3c6c-c6be-4cab-96d7-4b60e3d90214 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:56.234592 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d3c6c-c6be-4cab-96d7-4b60e3d90214 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:56.298360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4ccc3-ac1a-46fe-a024-99fb14bf89e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:56.298988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4ccc3-ac1a-46fe-a024-99fb14bf89e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:56.299240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4ccc3-ac1a-46fe-a024-99fb14bf89e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:56.299512 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4ccc3-ac1a-46fe-a024-99fb14bf89e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.300173 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4ccc3-ac1a-46fe-a024-99fb14bf89e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:56.302251 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4ccc3-ac1a-46fe-a024-99fb14bf89e2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:56.320477 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:56.328030 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1311/2630] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:56 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:56.332616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21fdce-77b4-4043-a52e-dddaa734b273 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:56.344603 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21fdce-77b4-4043-a52e-dddaa734b273 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:56.349466 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eef605-15af-4d81-b2cb-afb6d0804123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:56.350091 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eef605-15af-4d81-b2cb-afb6d0804123 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:56.350365 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eef605-15af-4d81-b2cb-afb6d0804123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:56.350633 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eef605-15af-4d81-b2cb-afb6d0804123 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.351353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eef605-15af-4d81-b2cb-afb6d0804123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:56.352918 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eef605-15af-4d81-b2cb-afb6d0804123 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:56.363422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1320/2631] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:56.368817 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4ccc3-ac1a-46fe-a024-99fb14bf89e2 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:56.379740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4ccc3-ac1a-46fe-a024-99fb14bf89e2 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:56.385174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a88779-9692-4afa-909b-7264802d683a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:56.386286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a88779-9692-4afa-909b-7264802d683a admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:56.386286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a88779-9692-4afa-909b-7264802d683a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:56.386286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a88779-9692-4afa-909b-7264802d683a admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.391427 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a88779-9692-4afa-909b-7264802d683a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:56.391427 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a88779-9692-4afa-909b-7264802d683a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:56.399992 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1312/2632] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:56.405716 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eef605-15af-4d81-b2cb-afb6d0804123 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:56.418940 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eef605-15af-4d81-b2cb-afb6d0804123 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:56.427049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc2ee6-1caa-4fdb-b103-074eedc053c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:56.427739 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc2ee6-1caa-4fdb-b103-074eedc053c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:56.428013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc2ee6-1caa-4fdb-b103-074eedc053c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:56.428284 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc2ee6-1caa-4fdb-b103-074eedc053c4 admin admin] PATH_INFO: `/v3/projects/a7485dc9fb364a259f7435bdca76d575/users/866651018e2f410a9161776299d5dbf8/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.443418 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc2ee6-1caa-4fdb-b103-074eedc053c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7485dc9fb364a259f7435bdca76d575, user_id=866651018e2f410a9161776299d5dbf8, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:56.446013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc2ee6-1caa-4fdb-b103-074eedc053c4 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:56.459992 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1321/2633] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:56 2022] PUT /identity/v3/projects/a7485dc9fb364a259f7435bdca76d575/users/866651018e2f410a9161776299d5dbf8/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:56.470924 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ac76b-63fe-4086-9217-27751ec66a77 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:56.471386 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ac76b-63fe-4086-9217-27751ec66a77 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:56.471887 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ac76b-63fe-4086-9217-27751ec66a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.501206 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-144ac76b-63fe-4086-9217-27751ec66a77 None None] MFA Rules not processed for user `866651018e2f410a9161776299d5dbf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:56.502553 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-144ac76b-63fe-4086-9217-27751ec66a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:56.574806 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1313/2634] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:56.580015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc2ee6-1caa-4fdb-b103-074eedc053c4 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:56.645587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc2ee6-1caa-4fdb-b103-074eedc053c4 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:56.650948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb5859-0b6f-456b-b8dc-5a0ffd221db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:56.651569 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb5859-0b6f-456b-b8dc-5a0ffd221db3 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:56.651816 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb5859-0b6f-456b-b8dc-5a0ffd221db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:56.652101 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb5859-0b6f-456b-b8dc-5a0ffd221db3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.652763 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb5859-0b6f-456b-b8dc-5a0ffd221db3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:56.654740 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb5859-0b6f-456b-b8dc-5a0ffd221db3 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:56.682766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1322/2635] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:56 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:56.687069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ac76b-63fe-4086-9217-27751ec66a77 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:56.699028 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ac76b-63fe-4086-9217-27751ec66a77 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:56.759311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a013b2-c987-4beb-93f8-39c213444b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:56.759976 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a013b2-c987-4beb-93f8-39c213444b43 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:56.760210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a013b2-c987-4beb-93f8-39c213444b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:56.760486 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a013b2-c987-4beb-93f8-39c213444b43 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.761190 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a013b2-c987-4beb-93f8-39c213444b43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:56.763166 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a013b2-c987-4beb-93f8-39c213444b43 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:56.786036 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1314/2636] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:56 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:56.790437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb5859-0b6f-456b-b8dc-5a0ffd221db3 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:56.804641 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb5859-0b6f-456b-b8dc-5a0ffd221db3 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:56.811856 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2c289-cdd9-4ae1-890d-78f01aa69f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:56.812718 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2c289-cdd9-4ae1-890d-78f01aa69f14 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:56.813152 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2c289-cdd9-4ae1-890d-78f01aa69f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:56.813586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2c289-cdd9-4ae1-890d-78f01aa69f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.814642 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2c289-cdd9-4ae1-890d-78f01aa69f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:56.817061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2c289-cdd9-4ae1-890d-78f01aa69f14 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:56.828645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1323/2637] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:56.832772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a013b2-c987-4beb-93f8-39c213444b43 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:56.843125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a013b2-c987-4beb-93f8-39c213444b43 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:56.847800 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060275e-0be8-4e91-a199-844414f9db3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:56.848411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060275e-0be8-4e91-a199-844414f9db3c admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:56.848739 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060275e-0be8-4e91-a199-844414f9db3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:56.849010 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060275e-0be8-4e91-a199-844414f9db3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.849703 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060275e-0be8-4e91-a199-844414f9db3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:56.851356 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9060275e-0be8-4e91-a199-844414f9db3c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:56.861543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1315/2638] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:31:56.865842 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2c289-cdd9-4ae1-890d-78f01aa69f14 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:56.876555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2c289-cdd9-4ae1-890d-78f01aa69f14 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:56.881746 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f35c9f-50a1-4a0e-b726-47640d3529dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:56.882440 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f35c9f-50a1-4a0e-b726-47640d3529dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:56.882719 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f35c9f-50a1-4a0e-b726-47640d3529dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:56.883004 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f35c9f-50a1-4a0e-b726-47640d3529dc admin admin] PATH_INFO: `/v3/projects/0d0e3e1e3b81475ebfd59681f7fbf882/users/bc80b0d456ad46799be23bac8630a811/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.899204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f35c9f-50a1-4a0e-b726-47640d3529dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d0e3e1e3b81475ebfd59681f7fbf882, user_id=bc80b0d456ad46799be23bac8630a811, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:56.902142 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f35c9f-50a1-4a0e-b726-47640d3529dc admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:56.923552 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1324/2639] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:56 2022] PUT /identity/v3/projects/0d0e3e1e3b81475ebfd59681f7fbf882/users/bc80b0d456ad46799be23bac8630a811/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:56.943553 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6bef6a-9cdd-44e5-966a-190eae102925 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:56.943844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6bef6a-9cdd-44e5-966a-190eae102925 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:56.944125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6bef6a-9cdd-44e5-966a-190eae102925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.975475 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-8f6bef6a-9cdd-44e5-966a-190eae102925 None None] MFA Rules not processed for user `bc80b0d456ad46799be23bac8630a811`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:56.976632 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8f6bef6a-9cdd-44e5-966a-190eae102925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:57.040241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:57.066821 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1316/2640] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:57.071707 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f35c9f-50a1-4a0e-b726-47640d3529dc admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:57.145896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f35c9f-50a1-4a0e-b726-47640d3529dc admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:57.151866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a5b4b-df80-403b-87bd-8a0687a1c592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:57.152508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a5b4b-df80-403b-87bd-8a0687a1c592 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:57.152778 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a5b4b-df80-403b-87bd-8a0687a1c592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:57.153050 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a5b4b-df80-403b-87bd-8a0687a1c592 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:57.153715 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a5b4b-df80-403b-87bd-8a0687a1c592 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:57.155701 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a5b4b-df80-403b-87bd-8a0687a1c592 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:57.160706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:57.188082 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1325/2641] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:31:57 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:57.192537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6bef6a-9cdd-44e5-966a-190eae102925 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:57.204543 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6bef6a-9cdd-44e5-966a-190eae102925 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:57.265151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1182f2-a1b6-44fc-ac51-3ddeac64166f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:57.265695 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1182f2-a1b6-44fc-ac51-3ddeac64166f admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:57.266032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1182f2-a1b6-44fc-ac51-3ddeac64166f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:57.266286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1182f2-a1b6-44fc-ac51-3ddeac64166f admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:57.266949 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1182f2-a1b6-44fc-ac51-3ddeac64166f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:57.268909 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1182f2-a1b6-44fc-ac51-3ddeac64166f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:57.291867 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1317/2642] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:31:57 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:31:57.295984 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a5b4b-df80-403b-87bd-8a0687a1c592 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:57.306858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a5b4b-df80-403b-87bd-8a0687a1c592 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:57.311704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e6327-a07a-4112-ab9d-57bdfd701ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:57.312319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e6327-a07a-4112-ab9d-57bdfd701ced admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:57.312598 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e6327-a07a-4112-ab9d-57bdfd701ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:57.312869 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e6327-a07a-4112-ab9d-57bdfd701ced admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:57.313577 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e6327-a07a-4112-ab9d-57bdfd701ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:57.315239 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e6327-a07a-4112-ab9d-57bdfd701ced admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:57.326088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1326/2643] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:57.330996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1182f2-a1b6-44fc-ac51-3ddeac64166f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:57.342317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1182f2-a1b6-44fc-ac51-3ddeac64166f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:57.348364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c843361-eca2-4c1f-8123-ea491b06bb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:57.349136 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c843361-eca2-4c1f-8123-ea491b06bb9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:57.349422 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c843361-eca2-4c1f-8123-ea491b06bb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:57.349805 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c843361-eca2-4c1f-8123-ea491b06bb9b admin admin] PATH_INFO: `/v3/projects/47e75e7dd99a4de496a03bc6f606c664/users/a418d1ac08634453815b5ff95f2ebf74/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:57.369749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c843361-eca2-4c1f-8123-ea491b06bb9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=47e75e7dd99a4de496a03bc6f606c664, user_id=a418d1ac08634453815b5ff95f2ebf74, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:57.373258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c843361-eca2-4c1f-8123-ea491b06bb9b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:57.393116 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1318/2644] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:57 2022] PUT /identity/v3/projects/47e75e7dd99a4de496a03bc6f606c664/users/a418d1ac08634453815b5ff95f2ebf74/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:57.397455 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e6327-a07a-4112-ab9d-57bdfd701ced admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:57.408031 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e6327-a07a-4112-ab9d-57bdfd701ced admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:57.468351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c70cc7-cafb-4f14-9ae6-aa41c43baf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:57.469013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c70cc7-cafb-4f14-9ae6-aa41c43baf69 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:57.469292 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c70cc7-cafb-4f14-9ae6-aa41c43baf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:57.469565 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c70cc7-cafb-4f14-9ae6-aa41c43baf69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:57.470308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c70cc7-cafb-4f14-9ae6-aa41c43baf69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:57.473255 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c70cc7-cafb-4f14-9ae6-aa41c43baf69 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:57.483215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1327/2645] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:31:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:31:57.487583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c843361-eca2-4c1f-8123-ea491b06bb9b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:57.500041 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c843361-eca2-4c1f-8123-ea491b06bb9b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:57.506100 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0fd28-f0e2-40cb-8923-d6e7c85ec81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:57.506788 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0fd28-f0e2-40cb-8923-d6e7c85ec81f admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:57.507083 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0fd28-f0e2-40cb-8923-d6e7c85ec81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:57.507345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0fd28-f0e2-40cb-8923-d6e7c85ec81f admin admin] PATH_INFO: `/v3/projects/47e75e7dd99a4de496a03bc6f606c664/users/a418d1ac08634453815b5ff95f2ebf74/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:57.509603 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0fd28-f0e2-40cb-8923-d6e7c85ec81f admin admin] RBAC: Authorizing `identity:create_grant(project_id=47e75e7dd99a4de496a03bc6f606c664, user_id=a418d1ac08634453815b5ff95f2ebf74, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:57.512308 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0fd28-f0e2-40cb-8923-d6e7c85ec81f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:57.524394 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1319/2646] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:31:57 2022] PUT /identity/v3/projects/47e75e7dd99a4de496a03bc6f606c664/users/a418d1ac08634453815b5ff95f2ebf74/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:57.537387 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aa61f-a76b-4797-91cb-e7f5d62bc1f7 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:57.537680 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aa61f-a76b-4797-91cb-e7f5d62bc1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:57.538106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aa61f-a76b-4797-91cb-e7f5d62bc1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:57.576367 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-119aa61f-a76b-4797-91cb-e7f5d62bc1f7 None None] MFA Rules not processed for user `a418d1ac08634453815b5ff95f2ebf74`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:31:57.577356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-119aa61f-a76b-4797-91cb-e7f5d62bc1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:57.659395 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1328/2647] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:31:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:31:57.671020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0fd28-f0e2-40cb-8923-d6e7c85ec81f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:57.673030 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-56a0fd28-f0e2-40cb-8923-d6e7c85ec81f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:57.735123 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0fd28-f0e2-40cb-8923-d6e7c85ec81f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:57.740433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa7f4c-739a-4117-9734-4a110c8972a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:57.741402 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa7f4c-739a-4117-9734-4a110c8972a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:57.741685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa7f4c-739a-4117-9734-4a110c8972a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:57.741984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa7f4c-739a-4117-9734-4a110c8972a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:57.755360 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa7f4c-739a-4117-9734-4a110c8972a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:57.758202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa7f4c-739a-4117-9734-4a110c8972a4 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:57.773486 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1320/2648] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:58.378183 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aa61f-a76b-4797-91cb-e7f5d62bc1f7 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:58.379780 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-119aa61f-a76b-4797-91cb-e7f5d62bc1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:58.392558 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aa61f-a76b-4797-91cb-e7f5d62bc1f7 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:58.396692 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a27ab3-d5e0-4110-b5cc-fd5a3d316957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:58.397351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a27ab3-d5e0-4110-b5cc-fd5a3d316957 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:58.397625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a27ab3-d5e0-4110-b5cc-fd5a3d316957 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:58.397954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a27ab3-d5e0-4110-b5cc-fd5a3d316957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:58.459572 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:58.463737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a27ab3-d5e0-4110-b5cc-fd5a3d316957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:58.466308 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a27ab3-d5e0-4110-b5cc-fd5a3d316957 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:58.477368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1329/2649] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:31:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:02.697162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa7f4c-739a-4117-9734-4a110c8972a4 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:02.706757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa7f4c-739a-4117-9734-4a110c8972a4 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:02.710979 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8b697-6ef9-402d-847f-444104229892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:02.711630 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8b697-6ef9-402d-847f-444104229892 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:02.711905 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8b697-6ef9-402d-847f-444104229892 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:02.712176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8b697-6ef9-402d-847f-444104229892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:02.755389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:02.777287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8b697-6ef9-402d-847f-444104229892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:02.778809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8b697-6ef9-402d-847f-444104229892 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:02.791873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1321/2650] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:05.535767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a27ab3-d5e0-4110-b5cc-fd5a3d316957 service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:05.537384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-a7a27ab3-d5e0-4110-b5cc-fd5a3d316957 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:05.610941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a27ab3-d5e0-4110-b5cc-fd5a3d316957 service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:05.615731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18389c0-ee75-49e1-bbcb-8ee3de038d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:05.616394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18389c0-ee75-49e1-bbcb-8ee3de038d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:05.616672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18389c0-ee75-49e1-bbcb-8ee3de038d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:05.616950 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18389c0-ee75-49e1-bbcb-8ee3de038d8b admin admin] PATH_INFO: `/v3/users/21129074968d40ff8f858401b8de755b` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:05.618172 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18389c0-ee75-49e1-bbcb-8ee3de038d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=21129074968d40ff8f858401b8de755b)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:05.620130 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18389c0-ee75-49e1-bbcb-8ee3de038d8b admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:05.678459 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a18389c0-ee75-49e1-bbcb-8ee3de038d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21129074968d40ff8f858401b8de755b'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:05.688364 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a18389c0-ee75-49e1-bbcb-8ee3de038d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21129074968d40ff8f858401b8de755b'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:05.689138 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a18389c0-ee75-49e1-bbcb-8ee3de038d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21129074968d40ff8f858401b8de755b'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:05.697533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a18389c0-ee75-49e1-bbcb-8ee3de038d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21129074968d40ff8f858401b8de755b'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:05.714039 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a18389c0-ee75-49e1-bbcb-8ee3de038d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21129074968d40ff8f858401b8de755b'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:05.721677 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1330/2651] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:05 2022] DELETE /identity/v3/users/21129074968d40ff8f858401b8de755b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:05.727023 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a8857-1565-4155-b200-7c7b9b2e4143 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:05.727500 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a8857-1565-4155-b200-7c7b9b2e4143 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:05.727500 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a8857-1565-4155-b200-7c7b9b2e4143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:05.748384 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-685a8857-1565-4155-b200-7c7b9b2e4143 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:05.749618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-685a8857-1565-4155-b200-7c7b9b2e4143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:05.813923 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1322/2652] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:05.824303 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18389c0-ee75-49e1-bbcb-8ee3de038d8b admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:05.825731 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-a18389c0-ee75-49e1-bbcb-8ee3de038d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:05.911965 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18389c0-ee75-49e1-bbcb-8ee3de038d8b admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:05.915899 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b768be8-6d75-4ddd-835f-f39a5ee4846c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:05.916583 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b768be8-6d75-4ddd-835f-f39a5ee4846c service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:05.916841 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b768be8-6d75-4ddd-835f-f39a5ee4846c service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:05.917111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b768be8-6d75-4ddd-835f-f39a5ee4846c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:05.925106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:05.928871 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b768be8-6d75-4ddd-835f-f39a5ee4846c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:05.931161 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b768be8-6d75-4ddd-835f-f39a5ee4846c service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:05.948445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1331/2653] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:06.249847 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a8857-1565-4155-b200-7c7b9b2e4143 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:06.251717 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-685a8857-1565-4155-b200-7c7b9b2e4143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:06.264821 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a8857-1565-4155-b200-7c7b9b2e4143 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:06.271243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6c5bc-6278-471d-8de3-8e95e403af22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:06.271957 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6c5bc-6278-471d-8de3-8e95e403af22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:06.272234 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6c5bc-6278-471d-8de3-8e95e403af22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:06.272505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6c5bc-6278-471d-8de3-8e95e403af22 admin admin] PATH_INFO: `/v3/projects/2942f199b22044a099a0160aa37197f6` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:06.273803 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6c5bc-6278-471d-8de3-8e95e403af22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2942f199b22044a099a0160aa37197f6)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:06.276404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f6c5bc-6278-471d-8de3-8e95e403af22 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:06.332874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-52f6c5bc-6278-471d-8de3-8e95e403af22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2942f199b22044a099a0160aa37197f6', 'request_id': 'req-52f6c5bc-6278-471d-8de3-8e95e403af22'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:06.342014 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-52f6c5bc-6278-471d-8de3-8e95e403af22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2942f199b22044a099a0160aa37197f6', 'request_id': 'req-52f6c5bc-6278-471d-8de3-8e95e403af22'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:06.351856 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-52f6c5bc-6278-471d-8de3-8e95e403af22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2942f199b22044a099a0160aa37197f6', 'request_id': 'req-52f6c5bc-6278-471d-8de3-8e95e403af22'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:06.368859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-52f6c5bc-6278-471d-8de3-8e95e403af22 admin admin] The token cache is being invalidate because project 2942f199b22044a099a0160aa37197f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:06.369234 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-52f6c5bc-6278-471d-8de3-8e95e403af22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:06.370054 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-52f6c5bc-6278-471d-8de3-8e95e403af22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:06.371839 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1323/2654] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:06 2022] DELETE /identity/v3/projects/2942f199b22044a099a0160aa37197f6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:06.376224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b768be8-6d75-4ddd-835f-f39a5ee4846c service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:06.387712 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b768be8-6d75-4ddd-835f-f39a5ee4846c service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:06.458408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66fbaf-a7b3-4c8e-bc8a-10aab7f3b2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:06.459164 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66fbaf-a7b3-4c8e-bc8a-10aab7f3b2ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:06.459446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66fbaf-a7b3-4c8e-bc8a-10aab7f3b2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:06.459722 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66fbaf-a7b3-4c8e-bc8a-10aab7f3b2ba admin admin] PATH_INFO: `/v3/users/db80c733c7bd4e8da0ccd756da987077` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:06.461053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66fbaf-a7b3-4c8e-bc8a-10aab7f3b2ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=db80c733c7bd4e8da0ccd756da987077)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:06.463357 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66fbaf-a7b3-4c8e-bc8a-10aab7f3b2ba admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:06.515513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-9f66fbaf-a7b3-4c8e-bc8a-10aab7f3b2ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db80c733c7bd4e8da0ccd756da987077'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:06.525268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-9f66fbaf-a7b3-4c8e-bc8a-10aab7f3b2ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db80c733c7bd4e8da0ccd756da987077'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:06.525954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-9f66fbaf-a7b3-4c8e-bc8a-10aab7f3b2ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db80c733c7bd4e8da0ccd756da987077'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:06.535113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-9f66fbaf-a7b3-4c8e-bc8a-10aab7f3b2ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db80c733c7bd4e8da0ccd756da987077'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:06.549840 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-9f66fbaf-a7b3-4c8e-bc8a-10aab7f3b2ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db80c733c7bd4e8da0ccd756da987077'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:06.558000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1332/2655] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:06 2022] DELETE /identity/v3/users/db80c733c7bd4e8da0ccd756da987077 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:06.671603 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6c5bc-6278-471d-8de3-8e95e403af22 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:06.673531 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-52f6c5bc-6278-471d-8de3-8e95e403af22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:06.688037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6c5bc-6278-471d-8de3-8e95e403af22 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:06.755370 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ea263-9504-4f48-8bdd-9ca358f19711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:06.756221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ea263-9504-4f48-8bdd-9ca358f19711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:06.756464 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ea263-9504-4f48-8bdd-9ca358f19711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:06.756809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ea263-9504-4f48-8bdd-9ca358f19711 admin admin] PATH_INFO: `/v3/users/866651018e2f410a9161776299d5dbf8` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:06.758107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ea263-9504-4f48-8bdd-9ca358f19711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=866651018e2f410a9161776299d5dbf8)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:06.760151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ea263-9504-4f48-8bdd-9ca358f19711 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:06.763092 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:06.832259 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-510ea263-9504-4f48-8bdd-9ca358f19711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '866651018e2f410a9161776299d5dbf8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:06.833018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-510ea263-9504-4f48-8bdd-9ca358f19711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '866651018e2f410a9161776299d5dbf8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:06.853700 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-510ea263-9504-4f48-8bdd-9ca358f19711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '866651018e2f410a9161776299d5dbf8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:06.866081 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-510ea263-9504-4f48-8bdd-9ca358f19711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '866651018e2f410a9161776299d5dbf8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:06.886518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-510ea263-9504-4f48-8bdd-9ca358f19711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '866651018e2f410a9161776299d5dbf8'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:06.888426 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66fbaf-a7b3-4c8e-bc8a-10aab7f3b2ba admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:06.891081 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9f66fbaf-a7b3-4c8e-bc8a-10aab7f3b2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:06.900466 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1324/2656] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:06 2022] DELETE /identity/v3/users/866651018e2f410a9161776299d5dbf8 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:06.905110 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11bfd6-2e8a-4368-b304-cc200ea1ab17 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:06.905503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11bfd6-2e8a-4368-b304-cc200ea1ab17 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:06.905711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11bfd6-2e8a-4368-b304-cc200ea1ab17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:06.914145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66fbaf-a7b3-4c8e-bc8a-10aab7f3b2ba admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:06.933475 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-cf11bfd6-2e8a-4368-b304-cc200ea1ab17 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:06.934880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cf11bfd6-2e8a-4368-b304-cc200ea1ab17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:07.015805 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486d8f7-3a2b-4bc5-a99d-c5b2d0b6ebcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:07.016588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486d8f7-3a2b-4bc5-a99d-c5b2d0b6ebcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:07.017001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486d8f7-3a2b-4bc5-a99d-c5b2d0b6ebcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:07.017623 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486d8f7-3a2b-4bc5-a99d-c5b2d0b6ebcb admin admin] PATH_INFO: `/v3/projects/9e9de624f55e4e9fbde35ef26ab16356` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:07.019012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486d8f7-3a2b-4bc5-a99d-c5b2d0b6ebcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e9de624f55e4e9fbde35ef26ab16356)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:07.020954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486d8f7-3a2b-4bc5-a99d-c5b2d0b6ebcb admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:07.042212 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1325/2657] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:07.051629 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:07.057426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11bfd6-2e8a-4368-b304-cc200ea1ab17 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:07.060046 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cf11bfd6-2e8a-4368-b304-cc200ea1ab17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:07.102440 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a486d8f7-3a2b-4bc5-a99d-c5b2d0b6ebcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e9de624f55e4e9fbde35ef26ab16356', 'request_id': 'req-a486d8f7-3a2b-4bc5-a99d-c5b2d0b6ebcb'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:07.111717 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a486d8f7-3a2b-4bc5-a99d-c5b2d0b6ebcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e9de624f55e4e9fbde35ef26ab16356', 'request_id': 'req-a486d8f7-3a2b-4bc5-a99d-c5b2d0b6ebcb'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:07.119225 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a486d8f7-3a2b-4bc5-a99d-c5b2d0b6ebcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e9de624f55e4e9fbde35ef26ab16356', 'request_id': 'req-a486d8f7-3a2b-4bc5-a99d-c5b2d0b6ebcb'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:07.132291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a486d8f7-3a2b-4bc5-a99d-c5b2d0b6ebcb admin admin] The token cache is being invalidate because project 9e9de624f55e4e9fbde35ef26ab16356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:07.132550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a486d8f7-3a2b-4bc5-a99d-c5b2d0b6ebcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:07.133359 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a486d8f7-3a2b-4bc5-a99d-c5b2d0b6ebcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:07.135504 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1333/2658] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:06 2022] DELETE /identity/v3/projects/9e9de624f55e4e9fbde35ef26ab16356 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:07.142027 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff40762-224d-4dac-ae97-ea2f3d3f9b32 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:07.142842 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff40762-224d-4dac-ae97-ea2f3d3f9b32 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:07.143479 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff40762-224d-4dac-ae97-ea2f3d3f9b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:07.144985 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11bfd6-2e8a-4368-b304-cc200ea1ab17 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:07.173725 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-5ff40762-224d-4dac-ae97-ea2f3d3f9b32 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:07.175009 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5ff40762-224d-4dac-ae97-ea2f3d3f9b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:07.186441 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:07.212162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d0cba-d438-468e-9930-29907e502f71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:07.212958 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d0cba-d438-468e-9930-29907e502f71 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:07.213262 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d0cba-d438-468e-9930-29907e502f71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:07.213564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d0cba-d438-468e-9930-29907e502f71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:07.215842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cd2d0cba-d438-468e-9930-29907e502f71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:07.247482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1334/2659] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:07.263921 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74683a26-29a3-4003-bf1e-efdb327abb6b None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:07.264193 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74683a26-29a3-4003-bf1e-efdb327abb6b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:07.264548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74683a26-29a3-4003-bf1e-efdb327abb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:07.287703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-74683a26-29a3-4003-bf1e-efdb327abb6b None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:07.289062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-74683a26-29a3-4003-bf1e-efdb327abb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:07.297433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d0cba-d438-468e-9930-29907e502f71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:07.299336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1335/2660] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:07.300229 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d0cba-d438-468e-9930-29907e502f71 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:07.304253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74683a26-29a3-4003-bf1e-efdb327abb6b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:07.313798 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1326/2661] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:32:07.314622 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74683a26-29a3-4003-bf1e-efdb327abb6b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:07.321542 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4fb0e-bfdc-4fc4-95c3-46d1faa59d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:07.322372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4fb0e-bfdc-4fc4-95c3-46d1faa59d0e admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:07.322655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4fb0e-bfdc-4fc4-95c3-46d1faa59d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:07.322951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4fb0e-bfdc-4fc4-95c3-46d1faa59d0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:07.323739 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4fb0e-bfdc-4fc4-95c3-46d1faa59d0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:32:07.324066 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4fb0e-bfdc-4fc4-95c3-46d1faa59d0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:07.325743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4fb0e-bfdc-4fc4-95c3-46d1faa59d0e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:07.339117 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1336/2662] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:32:07.343647 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d0cba-d438-468e-9930-29907e502f71 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:07.354962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d0cba-d438-468e-9930-29907e502f71 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:07.360185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3742fa2-069c-4afc-849c-e6ddb1a17a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:07.360851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3742fa2-069c-4afc-849c-e6ddb1a17a9a admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:07.361108 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3742fa2-069c-4afc-849c-e6ddb1a17a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:07.361401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3742fa2-069c-4afc-849c-e6ddb1a17a9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:07.362286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3742fa2-069c-4afc-849c-e6ddb1a17a9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:07.364165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3742fa2-069c-4afc-849c-e6ddb1a17a9a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:07.395536 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1327/2663] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:32:07 2022] POST /identity/v3/projects => generated 397 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:07.401058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4fb0e-bfdc-4fc4-95c3-46d1faa59d0e admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:07.413028 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4fb0e-bfdc-4fc4-95c3-46d1faa59d0e admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:07.490501 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634d425-55e2-40eb-bf2e-62a64970e5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:07.491235 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634d425-55e2-40eb-bf2e-62a64970e5e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:07.491708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634d425-55e2-40eb-bf2e-62a64970e5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:07.492128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634d425-55e2-40eb-bf2e-62a64970e5e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:07.493046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634d425-55e2-40eb-bf2e-62a64970e5e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:07.495126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634d425-55e2-40eb-bf2e-62a64970e5e1 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:07.525304 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1337/2664] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:32:07 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:07.529848 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3742fa2-069c-4afc-849c-e6ddb1a17a9a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:07.541593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3742fa2-069c-4afc-849c-e6ddb1a17a9a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:07.546760 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8a01a-4586-49da-92bc-0cfd3a2e37de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:07.547411 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8a01a-4586-49da-92bc-0cfd3a2e37de admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:07.547687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8a01a-4586-49da-92bc-0cfd3a2e37de admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:07.548045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8a01a-4586-49da-92bc-0cfd3a2e37de admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:07.548825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8a01a-4586-49da-92bc-0cfd3a2e37de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:07.550521 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8a01a-4586-49da-92bc-0cfd3a2e37de admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:07.563865 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1328/2665] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:07.568556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634d425-55e2-40eb-bf2e-62a64970e5e1 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:07.580219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634d425-55e2-40eb-bf2e-62a64970e5e1 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:07.585314 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17aaac-48c9-4757-8f63-099a43a3a232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:07.585985 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17aaac-48c9-4757-8f63-099a43a3a232 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:07.586265 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17aaac-48c9-4757-8f63-099a43a3a232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:07.586537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17aaac-48c9-4757-8f63-099a43a3a232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:07.587248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17aaac-48c9-4757-8f63-099a43a3a232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:07.588902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17aaac-48c9-4757-8f63-099a43a3a232 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:07.601807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1338/2666] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:07.606351 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8a01a-4586-49da-92bc-0cfd3a2e37de admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:07.620799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8a01a-4586-49da-92bc-0cfd3a2e37de admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:07.626765 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d245fa-4433-4f38-a1bb-20332e2adb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:07.627532 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d245fa-4433-4f38-a1bb-20332e2adb11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:07.627811 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d245fa-4433-4f38-a1bb-20332e2adb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:07.628311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d245fa-4433-4f38-a1bb-20332e2adb11 admin admin] PATH_INFO: `/v3/projects/ef88e73e5d304bb5855b0fee131b27ea/users/ab9d77f1f3674b789c1e955b759903d4/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:07.633587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17aaac-48c9-4757-8f63-099a43a3a232 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:07.635444 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-7d17aaac-48c9-4757-8f63-099a43a3a232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:07.646360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:07.651628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17aaac-48c9-4757-8f63-099a43a3a232 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:07.660746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d245fa-4433-4f38-a1bb-20332e2adb11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef88e73e5d304bb5855b0fee131b27ea, user_id=ab9d77f1f3674b789c1e955b759903d4, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:07.661403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240a503-e2e0-4d73-a6ee-2299f8c27085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:07.662273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240a503-e2e0-4d73-a6ee-2299f8c27085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:07.662571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240a503-e2e0-4d73-a6ee-2299f8c27085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:07.662854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240a503-e2e0-4d73-a6ee-2299f8c27085 admin admin] PATH_INFO: `/v3/projects/a7485dc9fb364a259f7435bdca76d575` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:07.665892 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d245fa-4433-4f38-a1bb-20332e2adb11 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:07.669649 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240a503-e2e0-4d73-a6ee-2299f8c27085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7485dc9fb364a259f7435bdca76d575)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:07.671723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240a503-e2e0-4d73-a6ee-2299f8c27085 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:07.680587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1329/2667] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:07 2022] PUT /identity/v3/projects/ef88e73e5d304bb5855b0fee131b27ea/users/ab9d77f1f3674b789c1e955b759903d4/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:07.744362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c240a503-e2e0-4d73-a6ee-2299f8c27085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7485dc9fb364a259f7435bdca76d575', 'request_id': 'req-c240a503-e2e0-4d73-a6ee-2299f8c27085'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:07.754978 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c240a503-e2e0-4d73-a6ee-2299f8c27085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7485dc9fb364a259f7435bdca76d575', 'request_id': 'req-c240a503-e2e0-4d73-a6ee-2299f8c27085'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:07.757268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c112da-6ad3-4f20-b294-d7d74f40a6a3 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:07.757762 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c112da-6ad3-4f20-b294-d7d74f40a6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:07.757926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c112da-6ad3-4f20-b294-d7d74f40a6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:07.764656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c240a503-e2e0-4d73-a6ee-2299f8c27085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7485dc9fb364a259f7435bdca76d575', 'request_id': 'req-c240a503-e2e0-4d73-a6ee-2299f8c27085'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:07.779057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c240a503-e2e0-4d73-a6ee-2299f8c27085 admin admin] The token cache is being invalidate because project a7485dc9fb364a259f7435bdca76d575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:07.779343 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c240a503-e2e0-4d73-a6ee-2299f8c27085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:07.780048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c240a503-e2e0-4d73-a6ee-2299f8c27085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:07.782300 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1339/2668] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:07 2022] DELETE /identity/v3/projects/a7485dc9fb364a259f7435bdca76d575 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:07.786454 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240a503-e2e0-4d73-a6ee-2299f8c27085 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:07.788085 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c240a503-e2e0-4d73-a6ee-2299f8c27085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:07.790241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-82c112da-6ad3-4f20-b294-d7d74f40a6a3 None None] MFA Rules not processed for user `ab9d77f1f3674b789c1e955b759903d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:07.791880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-82c112da-6ad3-4f20-b294-d7d74f40a6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:07.862305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240a503-e2e0-4d73-a6ee-2299f8c27085 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:07.871163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1330/2669] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:07.873261 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095459c-2aac-4710-8d38-042f4f08414f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:07.874498 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095459c-2aac-4710-8d38-042f4f08414f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:07.874969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095459c-2aac-4710-8d38-042f4f08414f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:07.875334 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095459c-2aac-4710-8d38-042f4f08414f admin admin] PATH_INFO: `/v3/users/bc80b0d456ad46799be23bac8630a811` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:07.875998 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c112da-6ad3-4f20-b294-d7d74f40a6a3 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:07.876790 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095459c-2aac-4710-8d38-042f4f08414f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc80b0d456ad46799be23bac8630a811)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:07.877501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-82c112da-6ad3-4f20-b294-d7d74f40a6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:07.878832 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095459c-2aac-4710-8d38-042f4f08414f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:07.893822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c112da-6ad3-4f20-b294-d7d74f40a6a3 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:07.898740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be6996-f31f-42d4-8886-cbd4bb9decf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:07.899371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be6996-f31f-42d4-8886-cbd4bb9decf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:07.899651 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be6996-f31f-42d4-8886-cbd4bb9decf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:07.899944 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be6996-f31f-42d4-8886-cbd4bb9decf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:07.900611 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be6996-f31f-42d4-8886-cbd4bb9decf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:07.902635 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be6996-f31f-42d4-8886-cbd4bb9decf1 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:07.930610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:07.941774 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1331/2670] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:32:07 2022] POST /identity/v3/projects => generated 399 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:32:07.946746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be6996-f31f-42d4-8886-cbd4bb9decf1 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:07.948109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d095459c-2aac-4710-8d38-042f4f08414f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc80b0d456ad46799be23bac8630a811'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:07.961154 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d095459c-2aac-4710-8d38-042f4f08414f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc80b0d456ad46799be23bac8630a811'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:07.962340 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d095459c-2aac-4710-8d38-042f4f08414f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc80b0d456ad46799be23bac8630a811'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:07.978863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d095459c-2aac-4710-8d38-042f4f08414f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc80b0d456ad46799be23bac8630a811'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:07.996823 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-d095459c-2aac-4710-8d38-042f4f08414f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc80b0d456ad46799be23bac8630a811'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:08.004382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1340/2671] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:07 2022] DELETE /identity/v3/users/bc80b0d456ad46799be23bac8630a811 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:08.039260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be6996-f31f-42d4-8886-cbd4bb9decf1 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:08.167713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b5451-cfcf-4b80-9369-448d188b0c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:08.168755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b5451-cfcf-4b80-9369-448d188b0c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:08.169549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b5451-cfcf-4b80-9369-448d188b0c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:08.170165 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b5451-cfcf-4b80-9369-448d188b0c33 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:08.171551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98b5451-cfcf-4b80-9369-448d188b0c33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:08.175557 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98b5451-cfcf-4b80-9369-448d188b0c33 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:08.211469 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1332/2672] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:32:07 2022] POST /identity/v3/users => generated 306 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:32:08.213032 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095459c-2aac-4710-8d38-042f4f08414f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:08.214659 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d095459c-2aac-4710-8d38-042f4f08414f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:08.234352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095459c-2aac-4710-8d38-042f4f08414f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:08.251370 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacc867-8200-4523-ad6f-24422e4cec7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:08.252058 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacc867-8200-4523-ad6f-24422e4cec7d admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:08.253600 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacc867-8200-4523-ad6f-24422e4cec7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:08.254005 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacc867-8200-4523-ad6f-24422e4cec7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:08.254768 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faacc867-8200-4523-ad6f-24422e4cec7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:08.256482 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faacc867-8200-4523-ad6f-24422e4cec7d admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:08.259242 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:08.273059 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1341/2673] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:08.278526 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b5451-cfcf-4b80-9369-448d188b0c33 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:08.295335 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98b5451-cfcf-4b80-9369-448d188b0c33 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:08.304517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f2110-f839-45a8-ab56-388a1ef0c252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:08.305612 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f2110-f839-45a8-ab56-388a1ef0c252 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:08.306068 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f2110-f839-45a8-ab56-388a1ef0c252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:08.306437 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f2110-f839-45a8-ab56-388a1ef0c252 admin admin] PATH_INFO: `/v3/projects/f2936df3c8cb49868cdfd895fe49967d/users/ddb013d21d77491f8fff946e24502d06/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:08.328317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f2110-f839-45a8-ab56-388a1ef0c252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2936df3c8cb49868cdfd895fe49967d, user_id=ddb013d21d77491f8fff946e24502d06, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:08.332394 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f2110-f839-45a8-ab56-388a1ef0c252 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:08.352430 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1333/2674] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:08 2022] PUT /identity/v3/projects/f2936df3c8cb49868cdfd895fe49967d/users/ddb013d21d77491f8fff946e24502d06/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:08.357989 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacc867-8200-4523-ad6f-24422e4cec7d admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:08.373473 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacc867-8200-4523-ad6f-24422e4cec7d admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:08.405257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f2110-f839-45a8-ab56-388a1ef0c252 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:08.408169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-960f2110-f839-45a8-ab56-388a1ef0c252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:08.465479 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb17560-2840-4cdd-861e-b52d0c9c4b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:08.466621 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb17560-2840-4cdd-861e-b52d0c9c4b72 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:08.466621 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb17560-2840-4cdd-861e-b52d0c9c4b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:08.466850 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb17560-2840-4cdd-861e-b52d0c9c4b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:08.467550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb17560-2840-4cdd-861e-b52d0c9c4b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:08.469188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb17560-2840-4cdd-861e-b52d0c9c4b72 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:08.481503 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1342/2675] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:08.487655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb17560-2840-4cdd-861e-b52d0c9c4b72 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:08.499391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb17560-2840-4cdd-861e-b52d0c9c4b72 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:08.506170 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0064909-7e09-4b6a-a603-8fc4b98245d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:08.506877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0064909-7e09-4b6a-a603-8fc4b98245d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:08.507183 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0064909-7e09-4b6a-a603-8fc4b98245d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:08.507457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0064909-7e09-4b6a-a603-8fc4b98245d9 admin admin] PATH_INFO: `/v3/projects/f2936df3c8cb49868cdfd895fe49967d/users/ddb013d21d77491f8fff946e24502d06/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:08.509929 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0064909-7e09-4b6a-a603-8fc4b98245d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2936df3c8cb49868cdfd895fe49967d, user_id=ddb013d21d77491f8fff946e24502d06, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:08.511053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f2110-f839-45a8-ab56-388a1ef0c252 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:08.513098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0064909-7e09-4b6a-a603-8fc4b98245d9 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:08.518467 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e1450-c7ac-4543-ac01-15e9abda0af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:08.519329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e1450-c7ac-4543-ac01-15e9abda0af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:08.519726 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e1450-c7ac-4543-ac01-15e9abda0af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:08.520069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e1450-c7ac-4543-ac01-15e9abda0af5 admin admin] PATH_INFO: `/v3/projects/0d0e3e1e3b81475ebfd59681f7fbf882` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:08.521915 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e1450-c7ac-4543-ac01-15e9abda0af5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d0e3e1e3b81475ebfd59681f7fbf882)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:08.524194 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e1450-c7ac-4543-ac01-15e9abda0af5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:08.530012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1343/2676] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:08 2022] PUT /identity/v3/projects/f2936df3c8cb49868cdfd895fe49967d/users/ddb013d21d77491f8fff946e24502d06/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:08.541351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a404fae-8c61-437d-989d-bfeb931e4c07 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:08.541760 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a404fae-8c61-437d-989d-bfeb931e4c07 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:08.542216 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a404fae-8c61-437d-989d-bfeb931e4c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:08.542533 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:08.582837 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-4a404fae-8c61-437d-989d-bfeb931e4c07 None None] MFA Rules not processed for user `ddb013d21d77491f8fff946e24502d06`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:08.585147 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4a404fae-8c61-437d-989d-bfeb931e4c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:08.594753 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-272e1450-c7ac-4543-ac01-15e9abda0af5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d0e3e1e3b81475ebfd59681f7fbf882', 'request_id': 'req-272e1450-c7ac-4543-ac01-15e9abda0af5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:08.601620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-272e1450-c7ac-4543-ac01-15e9abda0af5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d0e3e1e3b81475ebfd59681f7fbf882', 'request_id': 'req-272e1450-c7ac-4543-ac01-15e9abda0af5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:08.611446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-272e1450-c7ac-4543-ac01-15e9abda0af5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d0e3e1e3b81475ebfd59681f7fbf882', 'request_id': 'req-272e1450-c7ac-4543-ac01-15e9abda0af5'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:08.625554 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-272e1450-c7ac-4543-ac01-15e9abda0af5 admin admin] The token cache is being invalidate because project 0d0e3e1e3b81475ebfd59681f7fbf882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:08.625870 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-272e1450-c7ac-4543-ac01-15e9abda0af5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:08.626586 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-272e1450-c7ac-4543-ac01-15e9abda0af5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:08.628221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1334/2677] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:08 2022] DELETE /identity/v3/projects/0d0e3e1e3b81475ebfd59681f7fbf882 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:08.636927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e1450-c7ac-4543-ac01-15e9abda0af5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:08.638484 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-272e1450-c7ac-4543-ac01-15e9abda0af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:08.686523 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1344/2678] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:08 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:08.700474 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a404fae-8c61-437d-989d-bfeb931e4c07 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:08.701920 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4a404fae-8c61-437d-989d-bfeb931e4c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:08.730811 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e1450-c7ac-4543-ac01-15e9abda0af5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:08.736202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8871603-5947-4c2f-9281-63de85dd0473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:08.736928 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8871603-5947-4c2f-9281-63de85dd0473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:08.737208 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8871603-5947-4c2f-9281-63de85dd0473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:08.737829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8871603-5947-4c2f-9281-63de85dd0473 admin admin] PATH_INFO: `/v3/users/a418d1ac08634453815b5ff95f2ebf74` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:08.739405 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8871603-5947-4c2f-9281-63de85dd0473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a418d1ac08634453815b5ff95f2ebf74)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:08.741613 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8871603-5947-4c2f-9281-63de85dd0473 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:08.794541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:08.807157 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a404fae-8c61-437d-989d-bfeb931e4c07 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:08.813502 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d8871603-5947-4c2f-9281-63de85dd0473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a418d1ac08634453815b5ff95f2ebf74'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:08.814495 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6b570-0974-45a4-9309-2f7a9bd915c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:08.814707 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d8871603-5947-4c2f-9281-63de85dd0473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a418d1ac08634453815b5ff95f2ebf74'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:08.815367 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6b570-0974-45a4-9309-2f7a9bd915c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:08.815650 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6b570-0974-45a4-9309-2f7a9bd915c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:08.815922 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6b570-0974-45a4-9309-2f7a9bd915c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:08.829588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6b570-0974-45a4-9309-2f7a9bd915c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:08.831120 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a6b570-0974-45a4-9309-2f7a9bd915c0 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:08.832680 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d8871603-5947-4c2f-9281-63de85dd0473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a418d1ac08634453815b5ff95f2ebf74'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:08.843597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1345/2679] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:08 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:08.845807 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d8871603-5947-4c2f-9281-63de85dd0473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a418d1ac08634453815b5ff95f2ebf74'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:08.855506 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d8871603-5947-4c2f-9281-63de85dd0473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a418d1ac08634453815b5ff95f2ebf74'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:08.864336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1335/2680] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:08 2022] DELETE /identity/v3/users/a418d1ac08634453815b5ff95f2ebf74 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:08.950256 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6b570-0974-45a4-9309-2f7a9bd915c0 service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:08.961842 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6b570-0974-45a4-9309-2f7a9bd915c0 service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:09.037493 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694778a-e9d6-436a-8401-af1851d4f00d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:09.038178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694778a-e9d6-436a-8401-af1851d4f00d service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:09.038459 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694778a-e9d6-436a-8401-af1851d4f00d service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:09.039144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694778a-e9d6-436a-8401-af1851d4f00d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:09.040769 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-0694778a-e9d6-436a-8401-af1851d4f00d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:09.135945 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694778a-e9d6-436a-8401-af1851d4f00d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:09.137652 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694778a-e9d6-436a-8401-af1851d4f00d service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:09.154066 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1346/2681] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:08 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:09.242536 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8871603-5947-4c2f-9281-63de85dd0473 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:09.247028 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d8871603-5947-4c2f-9281-63de85dd0473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:09.299442 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:09.396890 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8871603-5947-4c2f-9281-63de85dd0473 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:09.402722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dc472-c755-44f0-a8a7-853510376040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:09.403507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dc472-c755-44f0-a8a7-853510376040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:09.403906 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dc472-c755-44f0-a8a7-853510376040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:09.404287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dc472-c755-44f0-a8a7-853510376040 admin admin] PATH_INFO: `/v3/projects/47e75e7dd99a4de496a03bc6f606c664` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:09.405851 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9dc472-c755-44f0-a8a7-853510376040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47e75e7dd99a4de496a03bc6f606c664)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:09.408170 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9dc472-c755-44f0-a8a7-853510376040 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:09.486231 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3a9dc472-c755-44f0-a8a7-853510376040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47e75e7dd99a4de496a03bc6f606c664', 'request_id': 'req-3a9dc472-c755-44f0-a8a7-853510376040'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:09.497372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3a9dc472-c755-44f0-a8a7-853510376040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47e75e7dd99a4de496a03bc6f606c664', 'request_id': 'req-3a9dc472-c755-44f0-a8a7-853510376040'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:09.508668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3a9dc472-c755-44f0-a8a7-853510376040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47e75e7dd99a4de496a03bc6f606c664', 'request_id': 'req-3a9dc472-c755-44f0-a8a7-853510376040'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:09.525032 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3a9dc472-c755-44f0-a8a7-853510376040 admin admin] The token cache is being invalidate because project 47e75e7dd99a4de496a03bc6f606c664 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:09.525418 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3a9dc472-c755-44f0-a8a7-853510376040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:09.526180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3a9dc472-c755-44f0-a8a7-853510376040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:09.528178 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1336/2682] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:09 2022] DELETE /identity/v3/projects/47e75e7dd99a4de496a03bc6f606c664 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:09.537854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2dc86-165d-4fe5-b3ac-fbe1ae4599f3 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:09.537854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2dc86-165d-4fe5-b3ac-fbe1ae4599f3 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:09.538369 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2dc86-165d-4fe5-b3ac-fbe1ae4599f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:09.567107 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-64f2dc86-165d-4fe5-b3ac-fbe1ae4599f3 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:09.568801 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-64f2dc86-165d-4fe5-b3ac-fbe1ae4599f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:09.689137 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1347/2683] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:09.698284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597d4ef-60f1-4311-84ac-efaa8a61c435 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:09.698513 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597d4ef-60f1-4311-84ac-efaa8a61c435 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:09.698637 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597d4ef-60f1-4311-84ac-efaa8a61c435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:09.726050 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7597d4ef-60f1-4311-84ac-efaa8a61c435 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:09.726896 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7597d4ef-60f1-4311-84ac-efaa8a61c435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:09.739583 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1337/2684] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:09.744784 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2dc86-165d-4fe5-b3ac-fbe1ae4599f3 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:09.755712 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2dc86-165d-4fe5-b3ac-fbe1ae4599f3 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:09.765238 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6085f2-fad4-46c8-99e8-27eb42cea86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:09.765802 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6085f2-fad4-46c8-99e8-27eb42cea86b admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:09.766092 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6085f2-fad4-46c8-99e8-27eb42cea86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:09.766362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6085f2-fad4-46c8-99e8-27eb42cea86b admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:09.767164 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6085f2-fad4-46c8-99e8-27eb42cea86b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:32:09.767467 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6085f2-fad4-46c8-99e8-27eb42cea86b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:09.769111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6085f2-fad4-46c8-99e8-27eb42cea86b admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:09.783733 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1348/2685] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:32:09.788910 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597d4ef-60f1-4311-84ac-efaa8a61c435 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:09.800818 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597d4ef-60f1-4311-84ac-efaa8a61c435 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:09.805913 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ca61d-967d-4403-ba9f-8615fa01038f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:09.806545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ca61d-967d-4403-ba9f-8615fa01038f admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:09.806823 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ca61d-967d-4403-ba9f-8615fa01038f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:09.807151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ca61d-967d-4403-ba9f-8615fa01038f admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:09.807762 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ca61d-967d-4403-ba9f-8615fa01038f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:09.809732 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ca61d-967d-4403-ba9f-8615fa01038f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:09.840498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1338/2686] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:32:09 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:09.848850 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6085f2-fad4-46c8-99e8-27eb42cea86b admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:09.866690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6085f2-fad4-46c8-99e8-27eb42cea86b admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:09.902644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:09.937727 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de011f65-9f56-4bfd-badf-e6ac90304d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:09.938504 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de011f65-9f56-4bfd-badf-e6ac90304d95 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:09.938784 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de011f65-9f56-4bfd-badf-e6ac90304d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:09.939063 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de011f65-9f56-4bfd-badf-e6ac90304d95 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:09.939702 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de011f65-9f56-4bfd-badf-e6ac90304d95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:09.941643 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de011f65-9f56-4bfd-badf-e6ac90304d95 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:09.967061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1349/2687] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:32:09 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:09.971600 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ca61d-967d-4403-ba9f-8615fa01038f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:09.985740 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ca61d-967d-4403-ba9f-8615fa01038f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:09.991436 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ad666-eaca-4885-ab64-f3c8bdfa2247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:09.992073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ad666-eaca-4885-ab64-f3c8bdfa2247 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:09.992344 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ad666-eaca-4885-ab64-f3c8bdfa2247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:09.992619 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ad666-eaca-4885-ab64-f3c8bdfa2247 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:09.993314 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ad666-eaca-4885-ab64-f3c8bdfa2247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:09.994964 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ad666-eaca-4885-ab64-f3c8bdfa2247 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.006389 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1339/2688] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:10.010713 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de011f65-9f56-4bfd-badf-e6ac90304d95 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.021851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de011f65-9f56-4bfd-badf-e6ac90304d95 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.026827 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204a7ba-ec79-489e-9d66-ced6c163a1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.027461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204a7ba-ec79-489e-9d66-ced6c163a1bd admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.027745 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204a7ba-ec79-489e-9d66-ced6c163a1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.028018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204a7ba-ec79-489e-9d66-ced6c163a1bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.028728 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204a7ba-ec79-489e-9d66-ced6c163a1bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.030398 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204a7ba-ec79-489e-9d66-ced6c163a1bd admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.042722 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1350/2689] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:10.047168 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ad666-eaca-4885-ab64-f3c8bdfa2247 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.059282 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ad666-eaca-4885-ab64-f3c8bdfa2247 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.064927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a0bf4-3d85-4ebd-b7bf-aadc9a1e9ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.065612 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a0bf4-3d85-4ebd-b7bf-aadc9a1e9ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.065901 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a0bf4-3d85-4ebd-b7bf-aadc9a1e9ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.066318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a0bf4-3d85-4ebd-b7bf-aadc9a1e9ac3 admin admin] PATH_INFO: `/v3/projects/2b7939d5e04c4f05b73b6afd84a382b9/users/a9e2f0f462bd44a094b550ae24144d71/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.083717 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a0bf4-3d85-4ebd-b7bf-aadc9a1e9ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b7939d5e04c4f05b73b6afd84a382b9, user_id=a9e2f0f462bd44a094b550ae24144d71, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.086557 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a0bf4-3d85-4ebd-b7bf-aadc9a1e9ac3 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.102257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1340/2690] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:10 2022] PUT /identity/v3/projects/2b7939d5e04c4f05b73b6afd84a382b9/users/a9e2f0f462bd44a094b550ae24144d71/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:10.115072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca198356-0046-43f7-bc26-f5b2283b775b None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.115357 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca198356-0046-43f7-bc26-f5b2283b775b None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.115633 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca198356-0046-43f7-bc26-f5b2283b775b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.148399 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-ca198356-0046-43f7-bc26-f5b2283b775b None None] MFA Rules not processed for user `a9e2f0f462bd44a094b550ae24144d71`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:10.149634 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-ca198356-0046-43f7-bc26-f5b2283b775b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:10.227506 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1351/2691] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:10.232183 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a0bf4-3d85-4ebd-b7bf-aadc9a1e9ac3 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.287673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:10.312615 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a0bf4-3d85-4ebd-b7bf-aadc9a1e9ac3 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.321515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c7dc5-42c1-4560-a830-b9e14a86f373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.322175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c7dc5-42c1-4560-a830-b9e14a86f373 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.322455 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c7dc5-42c1-4560-a830-b9e14a86f373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.322724 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c7dc5-42c1-4560-a830-b9e14a86f373 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.323412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c7dc5-42c1-4560-a830-b9e14a86f373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.325424 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c7dc5-42c1-4560-a830-b9e14a86f373 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.356319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1341/2692] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:32:10 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:10.360767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca198356-0046-43f7-bc26-f5b2283b775b None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.375508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca198356-0046-43f7-bc26-f5b2283b775b None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.433078 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c7dc5-42c1-4560-a830-b9e14a86f373 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.436295 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9e6c7dc5-42c1-4560-a830-b9e14a86f373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:10.447361 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15609f87-7b43-48e2-a1d8-8a2ec9e59e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.447361 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15609f87-7b43-48e2-a1d8-8a2ec9e59e7f admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.447361 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15609f87-7b43-48e2-a1d8-8a2ec9e59e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.447794 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15609f87-7b43-48e2-a1d8-8a2ec9e59e7f admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.448374 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15609f87-7b43-48e2-a1d8-8a2ec9e59e7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.450395 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15609f87-7b43-48e2-a1d8-8a2ec9e59e7f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.479359 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1352/2693] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:32:10 2022] POST /identity/v3/users => generated 300 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:10.483219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15609f87-7b43-48e2-a1d8-8a2ec9e59e7f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.494710 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15609f87-7b43-48e2-a1d8-8a2ec9e59e7f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.499564 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d26a7d-dd2a-46f7-a7a4-f6e4bced7b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.500198 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d26a7d-dd2a-46f7-a7a4-f6e4bced7b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.500478 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d26a7d-dd2a-46f7-a7a4-f6e4bced7b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.500755 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d26a7d-dd2a-46f7-a7a4-f6e4bced7b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.501462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d26a7d-dd2a-46f7-a7a4-f6e4bced7b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.503145 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d26a7d-dd2a-46f7-a7a4-f6e4bced7b0f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.513644 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1353/2694] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:32:10.515860 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c7dc5-42c1-4560-a830-b9e14a86f373 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.517802 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d26a7d-dd2a-46f7-a7a4-f6e4bced7b0f admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.520681 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e923256-80ae-47cd-8d78-6f18a68391c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.521427 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e923256-80ae-47cd-8d78-6f18a68391c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.521713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e923256-80ae-47cd-8d78-6f18a68391c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.522016 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e923256-80ae-47cd-8d78-6f18a68391c2 admin admin] PATH_INFO: `/v3/users/688d05a575ff4179a0344545c57c3cdb` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.523215 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e923256-80ae-47cd-8d78-6f18a68391c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=688d05a575ff4179a0344545c57c3cdb)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.525171 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e923256-80ae-47cd-8d78-6f18a68391c2 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.528571 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d26a7d-dd2a-46f7-a7a4-f6e4bced7b0f admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.533195 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526e22c-bdd8-40e0-9afe-3176c7f3c004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.533889 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526e22c-bdd8-40e0-9afe-3176c7f3c004 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.534169 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526e22c-bdd8-40e0-9afe-3176c7f3c004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.534439 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526e22c-bdd8-40e0-9afe-3176c7f3c004 admin admin] PATH_INFO: `/v3/projects/b25a2054467d47a7bd18b7b7e8b231ee/users/313d6430e5754d76ad7cb331538ff693/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.551078 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:10.557694 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526e22c-bdd8-40e0-9afe-3176c7f3c004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b25a2054467d47a7bd18b7b7e8b231ee, user_id=313d6430e5754d76ad7cb331538ff693, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.560631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526e22c-bdd8-40e0-9afe-3176c7f3c004 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.578946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5e923256-80ae-47cd-8d78-6f18a68391c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '688d05a575ff4179a0344545c57c3cdb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:10.579540 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5e923256-80ae-47cd-8d78-6f18a68391c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '688d05a575ff4179a0344545c57c3cdb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:10.583159 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1354/2695] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:10 2022] PUT /identity/v3/projects/b25a2054467d47a7bd18b7b7e8b231ee/users/313d6430e5754d76ad7cb331538ff693/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:10.587957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526e22c-bdd8-40e0-9afe-3176c7f3c004 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.593483 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5e923256-80ae-47cd-8d78-6f18a68391c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '688d05a575ff4179a0344545c57c3cdb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:10.600897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5e923256-80ae-47cd-8d78-6f18a68391c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '688d05a575ff4179a0344545c57c3cdb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:10.608508 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-5e923256-80ae-47cd-8d78-6f18a68391c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '688d05a575ff4179a0344545c57c3cdb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:10.628116 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1342/2696] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:10 2022] DELETE /identity/v3/users/688d05a575ff4179a0344545c57c3cdb => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:10.628116 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60870e-7bbb-40b9-997a-d9ea3f4a896d None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.628116 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60870e-7bbb-40b9-997a-d9ea3f4a896d None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.628116 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60870e-7bbb-40b9-997a-d9ea3f4a896d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.646613 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-de60870e-7bbb-40b9-997a-d9ea3f4a896d None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:10.648248 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-de60870e-7bbb-40b9-997a-d9ea3f4a896d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:10.669219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526e22c-bdd8-40e0-9afe-3176c7f3c004 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.713900 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:10.740184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1343/2697] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:10.747738 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60870e-7bbb-40b9-997a-d9ea3f4a896d None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.749176 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-de60870e-7bbb-40b9-997a-d9ea3f4a896d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:10.766013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e83495-17c5-48fb-8692-3918d409e6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.766013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e83495-17c5-48fb-8692-3918d409e6f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.766013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e83495-17c5-48fb-8692-3918d409e6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.766013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e83495-17c5-48fb-8692-3918d409e6f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.767231 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e83495-17c5-48fb-8692-3918d409e6f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.767231 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e83495-17c5-48fb-8692-3918d409e6f1 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.785597 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1355/2698] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:10.790598 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e83495-17c5-48fb-8692-3918d409e6f1 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.792560 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-96e83495-17c5-48fb-8692-3918d409e6f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:10.811716 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e83495-17c5-48fb-8692-3918d409e6f1 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.819705 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94478459-3056-4a32-bc4e-91814ca63dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.821041 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94478459-3056-4a32-bc4e-91814ca63dc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.821642 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94478459-3056-4a32-bc4e-91814ca63dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.822265 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94478459-3056-4a32-bc4e-91814ca63dc5 admin admin] PATH_INFO: `/v3/projects/b25a2054467d47a7bd18b7b7e8b231ee/users/313d6430e5754d76ad7cb331538ff693/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.826798 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60870e-7bbb-40b9-997a-d9ea3f4a896d None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.828601 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94478459-3056-4a32-bc4e-91814ca63dc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b25a2054467d47a7bd18b7b7e8b231ee, user_id=313d6430e5754d76ad7cb331538ff693, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.832474 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94478459-3056-4a32-bc4e-91814ca63dc5 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.836339 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115cb24-29cd-4c4d-be9b-169fb8f2f2b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.837535 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115cb24-29cd-4c4d-be9b-169fb8f2f2b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.839169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115cb24-29cd-4c4d-be9b-169fb8f2f2b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.839169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115cb24-29cd-4c4d-be9b-169fb8f2f2b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.861397 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1356/2699] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:10 2022] PUT /identity/v3/projects/b25a2054467d47a7bd18b7b7e8b231ee/users/313d6430e5754d76ad7cb331538ff693/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:10.862432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115cb24-29cd-4c4d-be9b-169fb8f2f2b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.867188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115cb24-29cd-4c4d-be9b-169fb8f2f2b0 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.872303 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa307471-3474-4296-a051-3d5864db3c16 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.872586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa307471-3474-4296-a051-3d5864db3c16 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.872863 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa307471-3474-4296-a051-3d5864db3c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.889480 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1344/2700] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:32:10.916599 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-aa307471-3474-4296-a051-3d5864db3c16 None None] MFA Rules not processed for user `313d6430e5754d76ad7cb331538ff693`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:10.918798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-aa307471-3474-4296-a051-3d5864db3c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:11.045161 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1357/2701] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:11.063278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115cb24-29cd-4c4d-be9b-169fb8f2f2b0 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:11.066133 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8115cb24-29cd-4c4d-be9b-169fb8f2f2b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:11.166572 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115cb24-29cd-4c4d-be9b-169fb8f2f2b0 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:11.225409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11787a48-cc60-4373-8070-bf7dd95e2af6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:11.226277 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11787a48-cc60-4373-8070-bf7dd95e2af6 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:11.226634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11787a48-cc60-4373-8070-bf7dd95e2af6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:11.227093 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11787a48-cc60-4373-8070-bf7dd95e2af6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:11.239758 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11787a48-cc60-4373-8070-bf7dd95e2af6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:11.241529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11787a48-cc60-4373-8070-bf7dd95e2af6 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:11.248911 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa307471-3474-4296-a051-3d5864db3c16 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:11.250479 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-aa307471-3474-4296-a051-3d5864db3c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:11.253362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1345/2702] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:32:11.262363 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:11.330876 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa307471-3474-4296-a051-3d5864db3c16 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:11.338743 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c4abe-ae16-4993-9331-c13c3dad0628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:11.340066 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c4abe-ae16-4993-9331-c13c3dad0628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:11.340429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c4abe-ae16-4993-9331-c13c3dad0628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:11.340711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c4abe-ae16-4993-9331-c13c3dad0628 admin admin] PATH_INFO: `/v3/projects/b1e48393e4bb4bb48ca444f86a07d29a` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:11.342237 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c4abe-ae16-4993-9331-c13c3dad0628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1e48393e4bb4bb48ca444f86a07d29a)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:11.345344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c4abe-ae16-4993-9331-c13c3dad0628 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:11.406253 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-9c8c4abe-ae16-4993-9331-c13c3dad0628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1e48393e4bb4bb48ca444f86a07d29a', 'request_id': 'req-9c8c4abe-ae16-4993-9331-c13c3dad0628'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:11.419088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-9c8c4abe-ae16-4993-9331-c13c3dad0628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1e48393e4bb4bb48ca444f86a07d29a', 'request_id': 'req-9c8c4abe-ae16-4993-9331-c13c3dad0628'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:11.420161 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11787a48-cc60-4373-8070-bf7dd95e2af6 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:11.422249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-9c8c4abe-ae16-4993-9331-c13c3dad0628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1e48393e4bb4bb48ca444f86a07d29a', 'request_id': 'req-9c8c4abe-ae16-4993-9331-c13c3dad0628'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:11.431680 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11787a48-cc60-4373-8070-bf7dd95e2af6 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:11.433586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-9c8c4abe-ae16-4993-9331-c13c3dad0628 admin admin] The token cache is being invalidate because project b1e48393e4bb4bb48ca444f86a07d29a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:11.433902 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-9c8c4abe-ae16-4993-9331-c13c3dad0628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:11.434556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-9c8c4abe-ae16-4993-9331-c13c3dad0628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:11.436284 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1358/2703] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:11 2022] DELETE /identity/v3/projects/b1e48393e4bb4bb48ca444f86a07d29a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:11.441023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f188527-715f-48d8-9ea9-188877cc0c3a None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:11.441304 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f188527-715f-48d8-9ea9-188877cc0c3a None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:11.441746 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f188527-715f-48d8-9ea9-188877cc0c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:11.465283 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-6f188527-715f-48d8-9ea9-188877cc0c3a None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:11.466580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6f188527-715f-48d8-9ea9-188877cc0c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:11.486797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:11.497064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198acc3-8148-4bb2-9153-3392559a6e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:11.497728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198acc3-8148-4bb2-9153-3392559a6e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:11.498069 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198acc3-8148-4bb2-9153-3392559a6e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:11.498307 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198acc3-8148-4bb2-9153-3392559a6e43 admin admin] PATH_INFO: `/v3/users/a9e2f0f462bd44a094b550ae24144d71` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:11.499511 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198acc3-8148-4bb2-9153-3392559a6e43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9e2f0f462bd44a094b550ae24144d71)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:11.501507 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198acc3-8148-4bb2-9153-3392559a6e43 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:11.530953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1359/2704] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:11.539754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5960b6-a98c-46fc-9198-aac710a615f2 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:11.540042 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5960b6-a98c-46fc-9198-aac710a615f2 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:11.540368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5960b6-a98c-46fc-9198-aac710a615f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:11.551512 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7198acc3-8148-4bb2-9153-3392559a6e43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9e2f0f462bd44a094b550ae24144d71'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:11.552171 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7198acc3-8148-4bb2-9153-3392559a6e43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9e2f0f462bd44a094b550ae24144d71'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:11.559555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-fe5960b6-a98c-46fc-9198-aac710a615f2 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:11.560766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-fe5960b6-a98c-46fc-9198-aac710a615f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:11.564538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7198acc3-8148-4bb2-9153-3392559a6e43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9e2f0f462bd44a094b550ae24144d71'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:11.571001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1360/2705] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:11.575101 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5960b6-a98c-46fc-9198-aac710a615f2 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:11.579855 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7198acc3-8148-4bb2-9153-3392559a6e43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9e2f0f462bd44a094b550ae24144d71'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:11.585735 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5960b6-a98c-46fc-9198-aac710a615f2 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:11.589380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7198acc3-8148-4bb2-9153-3392559a6e43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9e2f0f462bd44a094b550ae24144d71'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:11.592313 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed37f2-1d45-45e6-b17b-9fd0978ec67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:11.593000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed37f2-1d45-45e6-b17b-9fd0978ec67e admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:11.593274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed37f2-1d45-45e6-b17b-9fd0978ec67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:11.593542 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed37f2-1d45-45e6-b17b-9fd0978ec67e admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:11.594300 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed37f2-1d45-45e6-b17b-9fd0978ec67e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:32:11.594598 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed37f2-1d45-45e6-b17b-9fd0978ec67e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:11.596248 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed37f2-1d45-45e6-b17b-9fd0978ec67e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:11.597409 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1346/2706] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:11 2022] DELETE /identity/v3/users/a9e2f0f462bd44a094b550ae24144d71 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:11.602930 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9284f-155d-49a8-9fbf-96f77fdfccbb None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:11.602930 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9284f-155d-49a8-9fbf-96f77fdfccbb None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:11.602930 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9284f-155d-49a8-9fbf-96f77fdfccbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:11.612013 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1361/2707] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:32:11.616744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed37f2-1d45-45e6-b17b-9fd0978ec67e admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:11.619586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-cfed37f2-1d45-45e6-b17b-9fd0978ec67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:11.623746 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-9fb9284f-155d-49a8-9fbf-96f77fdfccbb None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:11.625515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9fb9284f-155d-49a8-9fbf-96f77fdfccbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:11.634917 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed37f2-1d45-45e6-b17b-9fd0978ec67e admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:11.649895 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:11.710894 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb5754-81eb-4757-b073-343f7c6b5a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:11.711575 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb5754-81eb-4757-b073-343f7c6b5a18 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:11.711850 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb5754-81eb-4757-b073-343f7c6b5a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:11.712112 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb5754-81eb-4757-b073-343f7c6b5a18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:11.712793 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb5754-81eb-4757-b073-343f7c6b5a18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:11.714225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1347/2708] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:11.715033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb5754-81eb-4757-b073-343f7c6b5a18 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:11.729203 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9284f-155d-49a8-9fbf-96f77fdfccbb None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:11.730706 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-9fb9284f-155d-49a8-9fbf-96f77fdfccbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:11.750624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1362/2709] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:32:11 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:11.753405 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb5754-81eb-4757-b073-343f7c6b5a18 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:11.796935 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9284f-155d-49a8-9fbf-96f77fdfccbb None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:11.825599 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:11.830900 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb5754-81eb-4757-b073-343f7c6b5a18 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:11.836337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a5999-1b59-4fd4-b70b-27dc6bc59df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:11.836991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a5999-1b59-4fd4-b70b-27dc6bc59df7 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:11.837268 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a5999-1b59-4fd4-b70b-27dc6bc59df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:11.837726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a5999-1b59-4fd4-b70b-27dc6bc59df7 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:11.838470 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a5999-1b59-4fd4-b70b-27dc6bc59df7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:11.840596 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a5999-1b59-4fd4-b70b-27dc6bc59df7 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:11.858138 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f3073-eaf2-4231-91f6-b0baeae53552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:11.858842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f3073-eaf2-4231-91f6-b0baeae53552 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:11.859240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f3073-eaf2-4231-91f6-b0baeae53552 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:11.859570 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f3073-eaf2-4231-91f6-b0baeae53552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:11.870107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f3073-eaf2-4231-91f6-b0baeae53552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:11.870967 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1363/2710] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:32:11 2022] POST /identity/v3/users => generated 291 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:11.871797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f3073-eaf2-4231-91f6-b0baeae53552 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:11.874915 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a5999-1b59-4fd4-b70b-27dc6bc59df7 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:11.882801 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1348/2711] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:32:11.884975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a5999-1b59-4fd4-b70b-27dc6bc59df7 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:11.889522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f45703-e01b-4972-bf7e-3a408bb91a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:11.890179 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f45703-e01b-4972-bf7e-3a408bb91a6a admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:11.890452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f45703-e01b-4972-bf7e-3a408bb91a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:11.890722 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f45703-e01b-4972-bf7e-3a408bb91a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:11.891429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f45703-e01b-4972-bf7e-3a408bb91a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:11.893062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f45703-e01b-4972-bf7e-3a408bb91a6a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:11.903741 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1364/2712] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:32:11.907846 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f3073-eaf2-4231-91f6-b0baeae53552 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:11.917847 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f3073-eaf2-4231-91f6-b0baeae53552 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:11.922785 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bef0a-6034-4e82-9858-31881c3c9e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:11.923431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bef0a-6034-4e82-9858-31881c3c9e7c admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:11.923713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bef0a-6034-4e82-9858-31881c3c9e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:11.923986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bef0a-6034-4e82-9858-31881c3c9e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:11.924669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49bef0a-6034-4e82-9858-31881c3c9e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:11.926634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49bef0a-6034-4e82-9858-31881c3c9e7c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:11.938344 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1349/2713] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:11.943108 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f45703-e01b-4972-bf7e-3a408bb91a6a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:11.955948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f45703-e01b-4972-bf7e-3a408bb91a6a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:11.961627 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc360c9d-2ecc-47a4-985b-e97445640e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:11.962400 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc360c9d-2ecc-47a4-985b-e97445640e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:11.962681 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc360c9d-2ecc-47a4-985b-e97445640e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:11.962953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc360c9d-2ecc-47a4-985b-e97445640e35 admin admin] PATH_INFO: `/v3/projects/6ffc20f69f0a4b3cbe7fc0ad2c2e17d6/users/d08fe46175654d8eb6ba0d47ae9110fa/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:11.979792 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc360c9d-2ecc-47a4-985b-e97445640e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffc20f69f0a4b3cbe7fc0ad2c2e17d6, user_id=d08fe46175654d8eb6ba0d47ae9110fa, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:11.982637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc360c9d-2ecc-47a4-985b-e97445640e35 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:11.996994 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1365/2714] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:11 2022] PUT /identity/v3/projects/6ffc20f69f0a4b3cbe7fc0ad2c2e17d6/users/d08fe46175654d8eb6ba0d47ae9110fa/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:12.008828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d233b1c-316e-4afb-9e68-d3584fc6d1be None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:12.009116 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d233b1c-316e-4afb-9e68-d3584fc6d1be None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:12.009388 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d233b1c-316e-4afb-9e68-d3584fc6d1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:12.046194 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3d233b1c-316e-4afb-9e68-d3584fc6d1be None None] MFA Rules not processed for user `d08fe46175654d8eb6ba0d47ae9110fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:12.048827 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3d233b1c-316e-4afb-9e68-d3584fc6d1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:12.122545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1350/2715] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:12.137585 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc360c9d-2ecc-47a4-985b-e97445640e35 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:12.140316 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-fc360c9d-2ecc-47a4-985b-e97445640e35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:12.198040 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d233b1c-316e-4afb-9e68-d3584fc6d1be None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:12.199576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3d233b1c-316e-4afb-9e68-d3584fc6d1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:12.237121 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc360c9d-2ecc-47a4-985b-e97445640e35 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:12.243037 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4b7a1-0a0f-4b16-97f6-c106f8089349 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:12.243788 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b7a1-0a0f-4b16-97f6-c106f8089349 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:12.244131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b7a1-0a0f-4b16-97f6-c106f8089349 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:12.244617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b7a1-0a0f-4b16-97f6-c106f8089349 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:12.254683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4b7a1-0a0f-4b16-97f6-c106f8089349 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:12.256389 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4b7a1-0a0f-4b16-97f6-c106f8089349 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:12.261951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:12.270190 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1366/2716] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:12.292648 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d233b1c-316e-4afb-9e68-d3584fc6d1be None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:12.298500 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d411082-1565-48f4-b1b6-c43f802f0b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:12.299155 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d411082-1565-48f4-b1b6-c43f802f0b86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:12.299442 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d411082-1565-48f4-b1b6-c43f802f0b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:12.299697 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d411082-1565-48f4-b1b6-c43f802f0b86 admin admin] PATH_INFO: `/v3/projects/2b7939d5e04c4f05b73b6afd84a382b9` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:12.300873 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d411082-1565-48f4-b1b6-c43f802f0b86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b7939d5e04c4f05b73b6afd84a382b9)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:12.303162 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d411082-1565-48f4-b1b6-c43f802f0b86 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:12.375863 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3d411082-1565-48f4-b1b6-c43f802f0b86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b7939d5e04c4f05b73b6afd84a382b9', 'request_id': 'req-3d411082-1565-48f4-b1b6-c43f802f0b86'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:12.378107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:12.383107 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3d411082-1565-48f4-b1b6-c43f802f0b86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b7939d5e04c4f05b73b6afd84a382b9', 'request_id': 'req-3d411082-1565-48f4-b1b6-c43f802f0b86'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:12.392266 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3d411082-1565-48f4-b1b6-c43f802f0b86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b7939d5e04c4f05b73b6afd84a382b9', 'request_id': 'req-3d411082-1565-48f4-b1b6-c43f802f0b86'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:12.413184 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3d411082-1565-48f4-b1b6-c43f802f0b86 admin admin] The token cache is being invalidate because project 2b7939d5e04c4f05b73b6afd84a382b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:12.413473 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3d411082-1565-48f4-b1b6-c43f802f0b86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:12.415393 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3d411082-1565-48f4-b1b6-c43f802f0b86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:12.418850 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1351/2717] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:12 2022] DELETE /identity/v3/projects/2b7939d5e04c4f05b73b6afd84a382b9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:12.424503 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4b7a1-0a0f-4b16-97f6-c106f8089349 service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:12.440435 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4b7a1-0a0f-4b16-97f6-c106f8089349 service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:12.509404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278dc56-3cc7-496e-99be-1b86b070e52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:12.510702 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278dc56-3cc7-496e-99be-1b86b070e52f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:12.510989 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278dc56-3cc7-496e-99be-1b86b070e52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:12.511278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278dc56-3cc7-496e-99be-1b86b070e52f admin admin] PATH_INFO: `/v3/users/313d6430e5754d76ad7cb331538ff693` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:12.512470 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278dc56-3cc7-496e-99be-1b86b070e52f admin admin] RBAC: Authorizing `identity:delete_user(user_id=313d6430e5754d76ad7cb331538ff693)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:12.514433 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278dc56-3cc7-496e-99be-1b86b070e52f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:12.583109 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a278dc56-3cc7-496e-99be-1b86b070e52f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '313d6430e5754d76ad7cb331538ff693'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:12.592827 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a278dc56-3cc7-496e-99be-1b86b070e52f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '313d6430e5754d76ad7cb331538ff693'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:12.594098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a278dc56-3cc7-496e-99be-1b86b070e52f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '313d6430e5754d76ad7cb331538ff693'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:12.604628 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a278dc56-3cc7-496e-99be-1b86b070e52f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '313d6430e5754d76ad7cb331538ff693'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:12.634285 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a278dc56-3cc7-496e-99be-1b86b070e52f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '313d6430e5754d76ad7cb331538ff693'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:12.640658 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1367/2718] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:12 2022] DELETE /identity/v3/users/313d6430e5754d76ad7cb331538ff693 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:12.932904 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d411082-1565-48f4-b1b6-c43f802f0b86 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:12.934459 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3d411082-1565-48f4-b1b6-c43f802f0b86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:12.953756 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d411082-1565-48f4-b1b6-c43f802f0b86 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:13.012843 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0654c33-cd42-4dbb-9053-e82994f4f8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:13.013529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0654c33-cd42-4dbb-9053-e82994f4f8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:13.013819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0654c33-cd42-4dbb-9053-e82994f4f8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:13.014091 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0654c33-cd42-4dbb-9053-e82994f4f8e1 admin admin] PATH_INFO: `/v3/projects/b25a2054467d47a7bd18b7b7e8b231ee` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:13.015311 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0654c33-cd42-4dbb-9053-e82994f4f8e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b25a2054467d47a7bd18b7b7e8b231ee)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:13.017236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0654c33-cd42-4dbb-9053-e82994f4f8e1 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:13.065060 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c0654c33-cd42-4dbb-9053-e82994f4f8e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b25a2054467d47a7bd18b7b7e8b231ee', 'request_id': 'req-c0654c33-cd42-4dbb-9053-e82994f4f8e1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:13.070816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c0654c33-cd42-4dbb-9053-e82994f4f8e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b25a2054467d47a7bd18b7b7e8b231ee', 'request_id': 'req-c0654c33-cd42-4dbb-9053-e82994f4f8e1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:13.079200 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c0654c33-cd42-4dbb-9053-e82994f4f8e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b25a2054467d47a7bd18b7b7e8b231ee', 'request_id': 'req-c0654c33-cd42-4dbb-9053-e82994f4f8e1'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:13.091516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c0654c33-cd42-4dbb-9053-e82994f4f8e1 admin admin] The token cache is being invalidate because project b25a2054467d47a7bd18b7b7e8b231ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:13.091835 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c0654c33-cd42-4dbb-9053-e82994f4f8e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:13.092569 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-c0654c33-cd42-4dbb-9053-e82994f4f8e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:13.094244 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1352/2719] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:12 2022] DELETE /identity/v3/projects/b25a2054467d47a7bd18b7b7e8b231ee => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:13.099408 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f1b3a-c7d9-4b09-b1c5-7ec5678c43f4 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:13.099878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f1b3a-c7d9-4b09-b1c5-7ec5678c43f4 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:13.100004 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f1b3a-c7d9-4b09-b1c5-7ec5678c43f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:13.120900 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-b57f1b3a-c7d9-4b09-b1c5-7ec5678c43f4 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:13.122221 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-b57f1b3a-c7d9-4b09-b1c5-7ec5678c43f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:13.185168 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1368/2720] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:13.195043 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017f7ff-7ad1-4f7e-9999-b7ed853aedb5 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:13.195347 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017f7ff-7ad1-4f7e-9999-b7ed853aedb5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:13.195712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017f7ff-7ad1-4f7e-9999-b7ed853aedb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:13.218390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3017f7ff-7ad1-4f7e-9999-b7ed853aedb5 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:13.219741 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3017f7ff-7ad1-4f7e-9999-b7ed853aedb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:13.229463 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1353/2721] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:13.234215 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f1b3a-c7d9-4b09-b1c5-7ec5678c43f4 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:13.244749 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f1b3a-c7d9-4b09-b1c5-7ec5678c43f4 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:13.250156 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06863a8-511f-428b-97d3-905eabd73433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:13.250781 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06863a8-511f-428b-97d3-905eabd73433 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:13.251057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06863a8-511f-428b-97d3-905eabd73433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:13.251324 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06863a8-511f-428b-97d3-905eabd73433 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:13.252361 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06863a8-511f-428b-97d3-905eabd73433 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:32:13.252754 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06863a8-511f-428b-97d3-905eabd73433 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:13.254438 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06863a8-511f-428b-97d3-905eabd73433 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:13.267564 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1369/2722] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:32:13.272755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017f7ff-7ad1-4f7e-9999-b7ed853aedb5 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:13.282209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017f7ff-7ad1-4f7e-9999-b7ed853aedb5 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:13.289601 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ed1d8-a116-4a6b-8d36-c24b46c90fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:13.290285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ed1d8-a116-4a6b-8d36-c24b46c90fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:13.290658 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ed1d8-a116-4a6b-8d36-c24b46c90fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:13.290973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ed1d8-a116-4a6b-8d36-c24b46c90fb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:13.291981 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ed1d8-a116-4a6b-8d36-c24b46c90fb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:13.295065 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ed1d8-a116-4a6b-8d36-c24b46c90fb9 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:13.335367 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1354/2723] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:32:13 2022] POST /identity/v3/projects => generated 409 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:13.339793 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06863a8-511f-428b-97d3-905eabd73433 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:13.350989 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06863a8-511f-428b-97d3-905eabd73433 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:13.354880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:13.425023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a0e72-7555-46dc-bab1-a19e2c89eb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:13.425651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a0e72-7555-46dc-bab1-a19e2c89eb08 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:13.425957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a0e72-7555-46dc-bab1-a19e2c89eb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:13.426229 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a0e72-7555-46dc-bab1-a19e2c89eb08 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:13.426899 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a0e72-7555-46dc-bab1-a19e2c89eb08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:13.428916 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a0e72-7555-46dc-bab1-a19e2c89eb08 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:13.457159 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1370/2724] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:32:13 2022] POST /identity/v3/users => generated 305 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:13.461730 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ed1d8-a116-4a6b-8d36-c24b46c90fb9 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:13.476980 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ed1d8-a116-4a6b-8d36-c24b46c90fb9 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:13.482059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df73f6f-5800-4e59-b2d8-f75d5edeea3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:13.482691 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df73f6f-5800-4e59-b2d8-f75d5edeea3e admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:13.482972 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df73f6f-5800-4e59-b2d8-f75d5edeea3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:13.483273 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df73f6f-5800-4e59-b2d8-f75d5edeea3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:13.483949 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df73f6f-5800-4e59-b2d8-f75d5edeea3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:13.485577 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df73f6f-5800-4e59-b2d8-f75d5edeea3e admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:13.499654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1355/2725] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:13.500890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a0e72-7555-46dc-bab1-a19e2c89eb08 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:13.511559 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a0e72-7555-46dc-bab1-a19e2c89eb08 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:13.517111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1516433-b3db-4620-a93f-be096be50511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:13.517752 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1516433-b3db-4620-a93f-be096be50511 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:13.518078 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1516433-b3db-4620-a93f-be096be50511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:13.518357 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1516433-b3db-4620-a93f-be096be50511 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:13.519052 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1516433-b3db-4620-a93f-be096be50511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:13.520704 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1516433-b3db-4620-a93f-be096be50511 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:13.533080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1371/2726] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:32:13.537729 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df73f6f-5800-4e59-b2d8-f75d5edeea3e admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:13.551545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df73f6f-5800-4e59-b2d8-f75d5edeea3e admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:13.556739 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a71dfc-da77-4fe5-b655-a537c00a40e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:13.557454 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a71dfc-da77-4fe5-b655-a537c00a40e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:13.557732 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a71dfc-da77-4fe5-b655-a537c00a40e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:13.558059 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a71dfc-da77-4fe5-b655-a537c00a40e5 admin admin] PATH_INFO: `/v3/projects/bc3951cdb6f246aaba44f91f0813bde4/users/240c6d10ab184f98a3f435e769e08146/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:13.574518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a71dfc-da77-4fe5-b655-a537c00a40e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc3951cdb6f246aaba44f91f0813bde4, user_id=240c6d10ab184f98a3f435e769e08146, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:13.577204 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a71dfc-da77-4fe5-b655-a537c00a40e5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:13.592493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1356/2727] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:13 2022] PUT /identity/v3/projects/bc3951cdb6f246aaba44f91f0813bde4/users/240c6d10ab184f98a3f435e769e08146/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:13.654933 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1516433-b3db-4620-a93f-be096be50511 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:13.658033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d1516433-b3db-4620-a93f-be096be50511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:13.665278 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b98e8-336e-4797-a26b-45897eed89de None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:13.665570 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b98e8-336e-4797-a26b-45897eed89de None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:13.665876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b98e8-336e-4797-a26b-45897eed89de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:13.679970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:13.690276 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1516433-b3db-4620-a93f-be096be50511 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:13.702178 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-567b98e8-336e-4797-a26b-45897eed89de None None] MFA Rules not processed for user `240c6d10ab184f98a3f435e769e08146`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:13.703985 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-567b98e8-336e-4797-a26b-45897eed89de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:13.761101 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584ce3f-31a2-4bed-9ec8-e9a011708bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:13.766286 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584ce3f-31a2-4bed-9ec8-e9a011708bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:13.766286 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584ce3f-31a2-4bed-9ec8-e9a011708bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:13.766286 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584ce3f-31a2-4bed-9ec8-e9a011708bbe admin admin] PATH_INFO: `/v3/users/ab9d77f1f3674b789c1e955b759903d4` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:13.766286 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a584ce3f-31a2-4bed-9ec8-e9a011708bbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab9d77f1f3674b789c1e955b759903d4)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:13.766286 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a584ce3f-31a2-4bed-9ec8-e9a011708bbe admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:13.785927 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1357/2728] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:13 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:13.788065 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b98e8-336e-4797-a26b-45897eed89de None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:13.800888 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567b98e8-336e-4797-a26b-45897eed89de None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:13.807321 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607992f-abf3-4f9d-9bb3-b633b83d2902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:13.808137 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607992f-abf3-4f9d-9bb3-b633b83d2902 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:13.808515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607992f-abf3-4f9d-9bb3-b633b83d2902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:13.809027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c607992f-abf3-4f9d-9bb3-b633b83d2902 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:13.809703 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607992f-abf3-4f9d-9bb3-b633b83d2902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:13.811758 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c607992f-abf3-4f9d-9bb3-b633b83d2902 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:13.825252 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a584ce3f-31a2-4bed-9ec8-e9a011708bbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab9d77f1f3674b789c1e955b759903d4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:13.837452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a584ce3f-31a2-4bed-9ec8-e9a011708bbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab9d77f1f3674b789c1e955b759903d4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:13.838537 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a584ce3f-31a2-4bed-9ec8-e9a011708bbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab9d77f1f3674b789c1e955b759903d4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:13.847549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1358/2729] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:32:13 2022] POST /identity/v3/projects => generated 409 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:32:13.849942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607992f-abf3-4f9d-9bb3-b633b83d2902 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:13.851345 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a584ce3f-31a2-4bed-9ec8-e9a011708bbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab9d77f1f3674b789c1e955b759903d4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:13.867206 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-a584ce3f-31a2-4bed-9ec8-e9a011708bbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab9d77f1f3674b789c1e955b759903d4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:13.876231 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1372/2730] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:13 2022] DELETE /identity/v3/users/ab9d77f1f3674b789c1e955b759903d4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:13.881889 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe10662-64b0-446d-9905-f093f4f1f1a2 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:13.882279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe10662-64b0-446d-9905-f093f4f1f1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:13.882648 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe10662-64b0-446d-9905-f093f4f1f1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:13.892184 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:13.907726 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-9fe10662-64b0-446d-9905-f093f4f1f1a2 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:13.910112 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9fe10662-64b0-446d-9905-f093f4f1f1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:13.933038 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c607992f-abf3-4f9d-9bb3-b633b83d2902 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:13.997213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1373/2731] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:13.998499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ced307-2c5d-4aab-a553-1e3f0baf4a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:13.998705 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe10662-64b0-446d-9905-f093f4f1f1a2 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:13.999140 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ced307-2c5d-4aab-a553-1e3f0baf4a8b admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:13.999417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ced307-2c5d-4aab-a553-1e3f0baf4a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:13.999744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ced307-2c5d-4aab-a553-1e3f0baf4a8b admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:14.000397 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ced307-2c5d-4aab-a553-1e3f0baf4a8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:14.001152 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9fe10662-64b0-446d-9905-f093f4f1f1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:14.002496 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ced307-2c5d-4aab-a553-1e3f0baf4a8b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:14.033498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1359/2732] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:32:13 2022] POST /identity/v3/users => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:32:14.034433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ced307-2c5d-4aab-a553-1e3f0baf4a8b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:14.036433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-00ced307-2c5d-4aab-a553-1e3f0baf4a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:14.050780 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ced307-2c5d-4aab-a553-1e3f0baf4a8b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:14.056268 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7f53e-1374-448a-9b85-18fa50580292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:14.056960 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7f53e-1374-448a-9b85-18fa50580292 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:14.057236 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7f53e-1374-448a-9b85-18fa50580292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:14.057554 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e7f53e-1374-448a-9b85-18fa50580292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:14.058522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7f53e-1374-448a-9b85-18fa50580292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:14.060515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e7f53e-1374-448a-9b85-18fa50580292 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:14.067440 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:14.078595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1360/2733] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:14.080721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7f53e-1374-448a-9b85-18fa50580292 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:14.082293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe10662-64b0-446d-9905-f093f4f1f1a2 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:14.087312 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08618f5a-7df2-4dac-900d-403162963532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:14.088123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08618f5a-7df2-4dac-900d-403162963532 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:14.088415 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08618f5a-7df2-4dac-900d-403162963532 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:14.089485 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08618f5a-7df2-4dac-900d-403162963532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:14.092693 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e7f53e-1374-448a-9b85-18fa50580292 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:14.097576 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ec537-d8bf-4bac-876c-fa47561c0e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:14.098284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ec537-d8bf-4bac-876c-fa47561c0e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:14.098586 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ec537-d8bf-4bac-876c-fa47561c0e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:14.098861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ec537-d8bf-4bac-876c-fa47561c0e30 admin admin] PATH_INFO: `/v3/projects/76c96cc1e5e9491aa0309ffad71ad326/users/b37e6a1f1a09401cb71cbea535b56641/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:14.099605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08618f5a-7df2-4dac-900d-403162963532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:14.101155 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08618f5a-7df2-4dac-900d-403162963532 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:14.114012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1374/2734] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:14.117925 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ec537-d8bf-4bac-876c-fa47561c0e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c96cc1e5e9491aa0309ffad71ad326, user_id=b37e6a1f1a09401cb71cbea535b56641, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:14.121039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ec537-d8bf-4bac-876c-fa47561c0e30 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:14.143785 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1361/2735] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:14 2022] PUT /identity/v3/projects/76c96cc1e5e9491aa0309ffad71ad326/users/b37e6a1f1a09401cb71cbea535b56641/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:14.148265 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08618f5a-7df2-4dac-900d-403162963532 service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:14.160772 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08618f5a-7df2-4dac-900d-403162963532 service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:14.234430 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da8cde-656d-4aa0-afc4-0fbf949b3ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:14.235061 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da8cde-656d-4aa0-afc4-0fbf949b3ead admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:14.235339 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da8cde-656d-4aa0-afc4-0fbf949b3ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:14.235606 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da8cde-656d-4aa0-afc4-0fbf949b3ead admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:14.236307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da8cde-656d-4aa0-afc4-0fbf949b3ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:14.237948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da8cde-656d-4aa0-afc4-0fbf949b3ead admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:14.248529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1375/2736] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:14.253849 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ec537-d8bf-4bac-876c-fa47561c0e30 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:14.265070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ec537-d8bf-4bac-876c-fa47561c0e30 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:14.281003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fc331-7396-42ca-8079-d748cfdb70db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:14.281696 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fc331-7396-42ca-8079-d748cfdb70db admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:14.282102 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fc331-7396-42ca-8079-d748cfdb70db admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:14.282262 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fc331-7396-42ca-8079-d748cfdb70db admin admin] PATH_INFO: `/v3/projects/76c96cc1e5e9491aa0309ffad71ad326/users/b37e6a1f1a09401cb71cbea535b56641/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:14.284249 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fc331-7396-42ca-8079-d748cfdb70db admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c96cc1e5e9491aa0309ffad71ad326, user_id=b37e6a1f1a09401cb71cbea535b56641, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:14.286993 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0fc331-7396-42ca-8079-d748cfdb70db admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:14.304881 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1362/2737] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:14 2022] PUT /identity/v3/projects/76c96cc1e5e9491aa0309ffad71ad326/users/b37e6a1f1a09401cb71cbea535b56641/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:14.317205 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf33a7c-e936-42b7-bf12-f040cc1b0d7e None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:14.317492 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf33a7c-e936-42b7-bf12-f040cc1b0d7e None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:14.317844 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf33a7c-e936-42b7-bf12-f040cc1b0d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:14.351846 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-aaf33a7c-e936-42b7-bf12-f040cc1b0d7e None None] MFA Rules not processed for user `b37e6a1f1a09401cb71cbea535b56641`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:14.353136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-aaf33a7c-e936-42b7-bf12-f040cc1b0d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:14.407003 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:14.441103 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fc331-7396-42ca-8079-d748cfdb70db admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:14.443271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-de0fc331-7396-42ca-8079-d748cfdb70db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:14.462483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1376/2738] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:14 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:14.478105 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf33a7c-e936-42b7-bf12-f040cc1b0d7e None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:14.484135 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-aaf33a7c-e936-42b7-bf12-f040cc1b0d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:14.545447 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0fc331-7396-42ca-8079-d748cfdb70db admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:14.550987 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:14.551680 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:14.551940 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:14.552247 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f admin admin] PATH_INFO: `/v3/projects/ef88e73e5d304bb5855b0fee131b27ea` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:14.553516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef88e73e5d304bb5855b0fee131b27ea)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:14.555501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:14.563941 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf33a7c-e936-42b7-bf12-f040cc1b0d7e None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:14.568516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af0cd4-5904-4fe0-b261-189e1fe2db01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:14.569191 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af0cd4-5904-4fe0-b261-189e1fe2db01 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:14.569464 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af0cd4-5904-4fe0-b261-189e1fe2db01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:14.569729 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af0cd4-5904-4fe0-b261-189e1fe2db01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:14.581033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af0cd4-5904-4fe0-b261-189e1fe2db01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:14.582545 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af0cd4-5904-4fe0-b261-189e1fe2db01 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:14.593696 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1377/2739] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:14 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:14.612164 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef88e73e5d304bb5855b0fee131b27ea', 'request_id': 'req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:14.618404 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef88e73e5d304bb5855b0fee131b27ea', 'request_id': 'req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:14.627876 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef88e73e5d304bb5855b0fee131b27ea', 'request_id': 'req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:14.642684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f admin admin] The token cache is being invalidate because project ef88e73e5d304bb5855b0fee131b27ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:14.642684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:14.642684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:14.645673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1363/2740] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:14 2022] DELETE /identity/v3/projects/ef88e73e5d304bb5855b0fee131b27ea => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:14.649683 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af0cd4-5904-4fe0-b261-189e1fe2db01 service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:14.661590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af0cd4-5904-4fe0-b261-189e1fe2db01 service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:14.740380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a3f94-dd56-47d4-9ff7-b06a79ff73ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:14.741152 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a3f94-dd56-47d4-9ff7-b06a79ff73ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:14.741503 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a3f94-dd56-47d4-9ff7-b06a79ff73ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:14.741878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a3f94-dd56-47d4-9ff7-b06a79ff73ac admin admin] PATH_INFO: `/v3/users/ddb013d21d77491f8fff946e24502d06` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:14.743458 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a3f94-dd56-47d4-9ff7-b06a79ff73ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddb013d21d77491f8fff946e24502d06)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:14.746598 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a3f94-dd56-47d4-9ff7-b06a79ff73ac admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:14.791973 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:14.793517 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:14.859646 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f6591-4fdd-4e31-8085-4fc3f17a451f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:14.864514 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e13c5c-50a7-4696-82a9-0f71dd7af4c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:14.865221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e13c5c-50a7-4696-82a9-0f71dd7af4c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:14.865468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e13c5c-50a7-4696-82a9-0f71dd7af4c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:14.865758 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e13c5c-50a7-4696-82a9-0f71dd7af4c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:14.867949 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c6e13c5c-50a7-4696-82a9-0f71dd7af4c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:14.877498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:14.939128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-877a3f94-dd56-47d4-9ff7-b06a79ff73ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddb013d21d77491f8fff946e24502d06'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:14.943653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e13c5c-50a7-4696-82a9-0f71dd7af4c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:14.945147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e13c5c-50a7-4696-82a9-0f71dd7af4c8 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:14.948367 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-877a3f94-dd56-47d4-9ff7-b06a79ff73ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddb013d21d77491f8fff946e24502d06'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:14.949099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-877a3f94-dd56-47d4-9ff7-b06a79ff73ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddb013d21d77491f8fff946e24502d06'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:14.956653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1364/2741] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:14 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:32:14.956653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e13c5c-50a7-4696-82a9-0f71dd7af4c8 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:14.957712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c6e13c5c-50a7-4696-82a9-0f71dd7af4c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:14.959326 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-877a3f94-dd56-47d4-9ff7-b06a79ff73ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddb013d21d77491f8fff946e24502d06'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:14.968913 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:14.971891 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e13c5c-50a7-4696-82a9-0f71dd7af4c8 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:14.976202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-877a3f94-dd56-47d4-9ff7-b06a79ff73ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddb013d21d77491f8fff946e24502d06'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:14.977871 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c4ece-23d3-4c0f-b04f-e8c38d87b7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:14.978432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c4ece-23d3-4c0f-b04f-e8c38d87b7e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:14.978711 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c4ece-23d3-4c0f-b04f-e8c38d87b7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:14.978984 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c4ece-23d3-4c0f-b04f-e8c38d87b7e6 admin admin] PATH_INFO: `/v3/users/d08fe46175654d8eb6ba0d47ae9110fa` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:14.980237 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c4ece-23d3-4c0f-b04f-e8c38d87b7e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d08fe46175654d8eb6ba0d47ae9110fa)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:14.982247 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c4ece-23d3-4c0f-b04f-e8c38d87b7e6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:14.985669 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1378/2742] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:14 2022] DELETE /identity/v3/users/ddb013d21d77491f8fff946e24502d06 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:15.040158 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e14c4ece-23d3-4c0f-b04f-e8c38d87b7e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd08fe46175654d8eb6ba0d47ae9110fa'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:15.041039 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e14c4ece-23d3-4c0f-b04f-e8c38d87b7e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd08fe46175654d8eb6ba0d47ae9110fa'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:15.059211 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e14c4ece-23d3-4c0f-b04f-e8c38d87b7e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd08fe46175654d8eb6ba0d47ae9110fa'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:15.071147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e14c4ece-23d3-4c0f-b04f-e8c38d87b7e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd08fe46175654d8eb6ba0d47ae9110fa'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:15.080515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-e14c4ece-23d3-4c0f-b04f-e8c38d87b7e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd08fe46175654d8eb6ba0d47ae9110fa'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:15.091505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1365/2743] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:14 2022] DELETE /identity/v3/users/d08fe46175654d8eb6ba0d47ae9110fa => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:15.097113 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b08768-4ba2-411d-ac2a-c73752d5330a None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:15.097329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b08768-4ba2-411d-ac2a-c73752d5330a None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:15.097677 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b08768-4ba2-411d-ac2a-c73752d5330a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:15.120854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-13b08768-4ba2-411d-ac2a-c73752d5330a None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:15.123015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-13b08768-4ba2-411d-ac2a-c73752d5330a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:15.213663 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1379/2744] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:15.224350 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c4ece-23d3-4c0f-b04f-e8c38d87b7e6 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:15.226522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e14c4ece-23d3-4c0f-b04f-e8c38d87b7e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:15.336347 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c4ece-23d3-4c0f-b04f-e8c38d87b7e6 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:15.341656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b08768-4ba2-411d-ac2a-c73752d5330a None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:15.342356 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af854c-8420-4af1-9db8-40f3272c817d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:15.343251 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-13b08768-4ba2-411d-ac2a-c73752d5330a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:15.343645 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af854c-8420-4af1-9db8-40f3272c817d service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:15.344151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af854c-8420-4af1-9db8-40f3272c817d service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:15.344595 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af854c-8420-4af1-9db8-40f3272c817d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:15.362368 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af854c-8420-4af1-9db8-40f3272c817d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:15.364364 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af854c-8420-4af1-9db8-40f3272c817d service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:15.366457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b08768-4ba2-411d-ac2a-c73752d5330a None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:15.377479 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1366/2745] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:32:15.379809 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4bbf8-17be-4e74-b363-e888c7bd5180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:15.381439 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4bbf8-17be-4e74-b363-e888c7bd5180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:15.381925 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4bbf8-17be-4e74-b363-e888c7bd5180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:15.383047 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4bbf8-17be-4e74-b363-e888c7bd5180 admin admin] PATH_INFO: `/v3/projects/f2936df3c8cb49868cdfd895fe49967d` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:15.385679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4bbf8-17be-4e74-b363-e888c7bd5180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2936df3c8cb49868cdfd895fe49967d)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:15.388556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4bbf8-17be-4e74-b363-e888c7bd5180 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:15.477502 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-02f4bbf8-17be-4e74-b363-e888c7bd5180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2936df3c8cb49868cdfd895fe49967d', 'request_id': 'req-02f4bbf8-17be-4e74-b363-e888c7bd5180'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:15.491444 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-02f4bbf8-17be-4e74-b363-e888c7bd5180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2936df3c8cb49868cdfd895fe49967d', 'request_id': 'req-02f4bbf8-17be-4e74-b363-e888c7bd5180'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:15.499881 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-02f4bbf8-17be-4e74-b363-e888c7bd5180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2936df3c8cb49868cdfd895fe49967d', 'request_id': 'req-02f4bbf8-17be-4e74-b363-e888c7bd5180'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:15.513375 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-02f4bbf8-17be-4e74-b363-e888c7bd5180 admin admin] The token cache is being invalidate because project f2936df3c8cb49868cdfd895fe49967d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:15.513951 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-02f4bbf8-17be-4e74-b363-e888c7bd5180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:15.514799 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-02f4bbf8-17be-4e74-b363-e888c7bd5180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:15.520001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1380/2746] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:15 2022] DELETE /identity/v3/projects/f2936df3c8cb49868cdfd895fe49967d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:15.521902 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c6f2b-653e-4793-b2a3-9bbdac1290da None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:15.522185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c6f2b-653e-4793-b2a3-9bbdac1290da None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:15.522457 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c6f2b-653e-4793-b2a3-9bbdac1290da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:15.543553 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-149c6f2b-653e-4793-b2a3-9bbdac1290da None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:15.544758 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-149c6f2b-653e-4793-b2a3-9bbdac1290da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:15.565130 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:15.616264 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1367/2747] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:15.626195 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392deb8a-b57b-4fb0-b281-73046af24f80 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:15.626663 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392deb8a-b57b-4fb0-b281-73046af24f80 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:15.627073 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392deb8a-b57b-4fb0-b281-73046af24f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:15.652265 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-392deb8a-b57b-4fb0-b281-73046af24f80 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:15.654453 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-392deb8a-b57b-4fb0-b281-73046af24f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:15.664178 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1381/2748] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:15.669019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c6f2b-653e-4793-b2a3-9bbdac1290da None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:15.681860 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c6f2b-653e-4793-b2a3-9bbdac1290da None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:15.688174 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d562b-ddf1-43a9-b364-7dd4ae368cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:15.689225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d562b-ddf1-43a9-b364-7dd4ae368cca admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:15.689597 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d562b-ddf1-43a9-b364-7dd4ae368cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:15.689919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d562b-ddf1-43a9-b364-7dd4ae368cca admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:15.690638 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d562b-ddf1-43a9-b364-7dd4ae368cca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:32:15.690962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d562b-ddf1-43a9-b364-7dd4ae368cca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:15.695018 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d562b-ddf1-43a9-b364-7dd4ae368cca admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:15.695807 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392deb8a-b57b-4fb0-b281-73046af24f80 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:15.697374 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-392deb8a-b57b-4fb0-b281-73046af24f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:15.709831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1368/2749] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:32:15.715050 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392deb8a-b57b-4fb0-b281-73046af24f80 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:15.718767 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d562b-ddf1-43a9-b364-7dd4ae368cca admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:15.722440 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b2c4dd-bca9-4fd1-af12-610631267c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:15.723275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2c4dd-bca9-4fd1-af12-610631267c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:15.723690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2c4dd-bca9-4fd1-af12-610631267c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:15.724186 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b2c4dd-bca9-4fd1-af12-610631267c03 admin admin] PATH_INFO: `/v3/projects/6ffc20f69f0a4b3cbe7fc0ad2c2e17d6` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:15.725725 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2c4dd-bca9-4fd1-af12-610631267c03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ffc20f69f0a4b3cbe7fc0ad2c2e17d6)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:15.729368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b2c4dd-bca9-4fd1-af12-610631267c03 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:15.731193 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d562b-ddf1-43a9-b364-7dd4ae368cca admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:15.737850 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05111d5d-df1c-4cce-9975-0ceba2665360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:15.738497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05111d5d-df1c-4cce-9975-0ceba2665360 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:15.738769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05111d5d-df1c-4cce-9975-0ceba2665360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:15.739034 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05111d5d-df1c-4cce-9975-0ceba2665360 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:15.739669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05111d5d-df1c-4cce-9975-0ceba2665360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:15.741708 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05111d5d-df1c-4cce-9975-0ceba2665360 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:15.771586 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1369/2750] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:32:15 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:15.775816 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05111d5d-df1c-4cce-9975-0ceba2665360 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:15.790498 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-b9b2c4dd-bca9-4fd1-af12-610631267c03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ffc20f69f0a4b3cbe7fc0ad2c2e17d6', 'request_id': 'req-b9b2c4dd-bca9-4fd1-af12-610631267c03'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:15.800035 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-b9b2c4dd-bca9-4fd1-af12-610631267c03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ffc20f69f0a4b3cbe7fc0ad2c2e17d6', 'request_id': 'req-b9b2c4dd-bca9-4fd1-af12-610631267c03'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:15.807071 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-b9b2c4dd-bca9-4fd1-af12-610631267c03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ffc20f69f0a4b3cbe7fc0ad2c2e17d6', 'request_id': 'req-b9b2c4dd-bca9-4fd1-af12-610631267c03'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:15.809937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:15.820738 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-b9b2c4dd-bca9-4fd1-af12-610631267c03 admin admin] The token cache is being invalidate because project 6ffc20f69f0a4b3cbe7fc0ad2c2e17d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:15.821023 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-b9b2c4dd-bca9-4fd1-af12-610631267c03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:15.822019 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-b9b2c4dd-bca9-4fd1-af12-610631267c03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:15.824368 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1382/2751] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:15 2022] DELETE /identity/v3/projects/6ffc20f69f0a4b3cbe7fc0ad2c2e17d6 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:15.829873 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddd584-1562-4b5d-88c8-957b6a4f7e92 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:15.830155 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddd584-1562-4b5d-88c8-957b6a4f7e92 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:15.830428 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddd584-1562-4b5d-88c8-957b6a4f7e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:15.849034 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05111d5d-df1c-4cce-9975-0ceba2665360 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:15.850839 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-01ddd584-1562-4b5d-88c8-957b6a4f7e92 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:15.852063 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-01ddd584-1562-4b5d-88c8-957b6a4f7e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:15.923890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1383/2752] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:15.925978 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72daf59-e3f7-4d19-ab21-8842b1abee01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:15.926618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72daf59-e3f7-4d19-ab21-8842b1abee01 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:15.926898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72daf59-e3f7-4d19-ab21-8842b1abee01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:15.927257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72daf59-e3f7-4d19-ab21-8842b1abee01 admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:15.928186 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72daf59-e3f7-4d19-ab21-8842b1abee01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:15.930261 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72daf59-e3f7-4d19-ab21-8842b1abee01 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:15.932115 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8635a-b0c5-4eee-b08b-c1d171b8af08 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:15.932393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8635a-b0c5-4eee-b08b-c1d171b8af08 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:15.932660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8635a-b0c5-4eee-b08b-c1d171b8af08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:15.953512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-cfc8635a-b0c5-4eee-b08b-c1d171b8af08 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:15.954380 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1370/2753] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:32:15 2022] POST /identity/v3/users => generated 291 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:32:15.954883 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-cfc8635a-b0c5-4eee-b08b-c1d171b8af08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:15.958898 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72daf59-e3f7-4d19-ab21-8842b1abee01 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:15.960895 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-a72daf59-e3f7-4d19-ab21-8842b1abee01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:15.963567 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1384/2754] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:15.967860 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8635a-b0c5-4eee-b08b-c1d171b8af08 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:15.973917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72daf59-e3f7-4d19-ab21-8842b1abee01 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:15.978530 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8635a-b0c5-4eee-b08b-c1d171b8af08 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:15.979049 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e76d0-6635-4124-a724-cfa350ac2c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:15.979699 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e76d0-6635-4124-a724-cfa350ac2c4f admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:15.979987 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e76d0-6635-4124-a724-cfa350ac2c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:15.980261 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e76d0-6635-4124-a724-cfa350ac2c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:15.981000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e76d0-6635-4124-a724-cfa350ac2c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:15.982668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e76d0-6635-4124-a724-cfa350ac2c4f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:15.983877 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbfdd8-aa4a-4188-b8c0-7de07ae3c6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:15.984813 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbfdd8-aa4a-4188-b8c0-7de07ae3c6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:15.985287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbfdd8-aa4a-4188-b8c0-7de07ae3c6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:15.985748 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cbfdd8-aa4a-4188-b8c0-7de07ae3c6d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:15.986651 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbfdd8-aa4a-4188-b8c0-7de07ae3c6d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:32:15.986956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbfdd8-aa4a-4188-b8c0-7de07ae3c6d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:15.988568 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cbfdd8-aa4a-4188-b8c0-7de07ae3c6d6 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:15.993995 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1371/2755] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:15.999310 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e76d0-6635-4124-a724-cfa350ac2c4f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:16.003885 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1385/2756] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:32:16.008131 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbfdd8-aa4a-4188-b8c0-7de07ae3c6d6 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:16.016303 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e76d0-6635-4124-a724-cfa350ac2c4f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:16.020690 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5c9be-b9ee-46ef-a0fe-470db5f57732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:16.021531 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5c9be-b9ee-46ef-a0fe-470db5f57732 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.021883 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5c9be-b9ee-46ef-a0fe-470db5f57732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.022293 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5c9be-b9ee-46ef-a0fe-470db5f57732 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.023456 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5c9be-b9ee-46ef-a0fe-470db5f57732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:16.025362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5c9be-b9ee-46ef-a0fe-470db5f57732 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:16.026167 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cbfdd8-aa4a-4188-b8c0-7de07ae3c6d6 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:16.033351 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604b1ee-2075-40de-8759-59f9fb3e7dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:16.034320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604b1ee-2075-40de-8759-59f9fb3e7dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.034605 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604b1ee-2075-40de-8759-59f9fb3e7dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.034884 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604b1ee-2075-40de-8759-59f9fb3e7dd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.035614 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604b1ee-2075-40de-8759-59f9fb3e7dd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:16.037163 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1372/2757] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:16.037703 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604b1ee-2075-40de-8759-59f9fb3e7dd9 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:16.042005 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5c9be-b9ee-46ef-a0fe-470db5f57732 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:16.048416 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:16.057538 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5c9be-b9ee-46ef-a0fe-470db5f57732 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:16.064468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e89c3-7dbd-4395-b159-6aa79d5a6c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:16.065238 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e89c3-7dbd-4395-b159-6aa79d5a6c01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.065515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e89c3-7dbd-4395-b159-6aa79d5a6c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.065809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e89c3-7dbd-4395-b159-6aa79d5a6c01 admin admin] PATH_INFO: `/v3/projects/ecd39538ef64401e831ff44fcae96962/users/dc5b1f13b8234e04b97236e66e2b21d9/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.070193 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1386/2758] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:32:16 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:32:16.074216 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604b1ee-2075-40de-8759-59f9fb3e7dd9 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:16.083546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e89c3-7dbd-4395-b159-6aa79d5a6c01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecd39538ef64401e831ff44fcae96962, user_id=dc5b1f13b8234e04b97236e66e2b21d9, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:16.086600 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e89c3-7dbd-4395-b159-6aa79d5a6c01 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:16.106717 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1373/2759] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:16 2022] PUT /identity/v3/projects/ecd39538ef64401e831ff44fcae96962/users/dc5b1f13b8234e04b97236e66e2b21d9/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:16.119335 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92f74d-25c2-4757-89be-fcc3972fcaed None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.119625 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92f74d-25c2-4757-89be-fcc3972fcaed None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.119933 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92f74d-25c2-4757-89be-fcc3972fcaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.148818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604b1ee-2075-40de-8759-59f9fb3e7dd9 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:16.153206 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-8a92f74d-25c2-4757-89be-fcc3972fcaed None None] MFA Rules not processed for user `dc5b1f13b8234e04b97236e66e2b21d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:16.154582 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-8a92f74d-25c2-4757-89be-fcc3972fcaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:16.209666 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:16.217732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1321494-460d-47a3-af86-0df0ec9f729d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:16.218428 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1321494-460d-47a3-af86-0df0ec9f729d admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.218970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1321494-460d-47a3-af86-0df0ec9f729d admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.219254 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1321494-460d-47a3-af86-0df0ec9f729d admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.220298 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1321494-460d-47a3-af86-0df0ec9f729d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:16.223015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1321494-460d-47a3-af86-0df0ec9f729d admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:16.248283 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1387/2760] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:32:16 2022] POST /identity/v3/users => generated 283 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:32:16.249341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1374/2761] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:16.253858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1321494-460d-47a3-af86-0df0ec9f729d admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:16.254091 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92f74d-25c2-4757-89be-fcc3972fcaed None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:16.265277 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1321494-460d-47a3-af86-0df0ec9f729d admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:16.266064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a92f74d-25c2-4757-89be-fcc3972fcaed None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:16.270512 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951fa28-8f6a-42da-905d-350d46519cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:16.271125 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951fa28-8f6a-42da-905d-350d46519cee admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.271326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee0494-cb47-41ae-8f80-2ae81fb34b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:16.271457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951fa28-8f6a-42da-905d-350d46519cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.271650 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951fa28-8f6a-42da-905d-350d46519cee admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.271968 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee0494-cb47-41ae-8f80-2ae81fb34b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.272250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee0494-cb47-41ae-8f80-2ae81fb34b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.272358 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951fa28-8f6a-42da-905d-350d46519cee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:16.272521 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee0494-cb47-41ae-8f80-2ae81fb34b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.273282 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee0494-cb47-41ae-8f80-2ae81fb34b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:16.274266 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951fa28-8f6a-42da-905d-350d46519cee admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:16.274942 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee0494-cb47-41ae-8f80-2ae81fb34b1b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:16.286202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1375/2762] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:32:16.290532 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee0494-cb47-41ae-8f80-2ae81fb34b1b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:16.303427 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee0494-cb47-41ae-8f80-2ae81fb34b1b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:16.307539 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1388/2763] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:32:16 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:16.308890 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81a0fc-11a1-4ba7-a4a3-417bdc77bb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:16.309537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81a0fc-11a1-4ba7-a4a3-417bdc77bb4f admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.309828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81a0fc-11a1-4ba7-a4a3-417bdc77bb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.310083 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81a0fc-11a1-4ba7-a4a3-417bdc77bb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.310877 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81a0fc-11a1-4ba7-a4a3-417bdc77bb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:16.312035 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951fa28-8f6a-42da-905d-350d46519cee admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:16.312563 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81a0fc-11a1-4ba7-a4a3-417bdc77bb4f admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:16.326879 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1376/2764] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:16.328355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81a0fc-11a1-4ba7-a4a3-417bdc77bb4f admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:16.340634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81a0fc-11a1-4ba7-a4a3-417bdc77bb4f admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:16.388615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951fa28-8f6a-42da-905d-350d46519cee admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:16.393409 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb9c4f-5806-4cee-80a6-d9aa77c6c63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:16.394069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb9c4f-5806-4cee-80a6-d9aa77c6c63b admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.394348 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb9c4f-5806-4cee-80a6-d9aa77c6c63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.394659 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb9c4f-5806-4cee-80a6-d9aa77c6c63b admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.395331 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb9c4f-5806-4cee-80a6-d9aa77c6c63b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:16.397595 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb9c4f-5806-4cee-80a6-d9aa77c6c63b admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:16.416038 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef86bb6-206a-4c28-8107-9350c3a2f30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:16.417017 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef86bb6-206a-4c28-8107-9350c3a2f30c admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.417353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef86bb6-206a-4c28-8107-9350c3a2f30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.417697 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef86bb6-206a-4c28-8107-9350c3a2f30c admin admin] PATH_INFO: `/v3/projects/5cdf1fff648c440eb3b69dc25b5d8dc6/users/782acde280984ec6a01d788c95754f35/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.423916 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1389/2765] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:32:16 2022] POST /identity/v3/users => generated 297 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:32:16.428054 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb9c4f-5806-4cee-80a6-d9aa77c6c63b admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:16.436210 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef86bb6-206a-4c28-8107-9350c3a2f30c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cdf1fff648c440eb3b69dc25b5d8dc6, user_id=782acde280984ec6a01d788c95754f35, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:16.438249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb9c4f-5806-4cee-80a6-d9aa77c6c63b admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:16.438995 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef86bb6-206a-4c28-8107-9350c3a2f30c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:16.443936 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7391afd-82e1-4cf8-b63f-416cfe3da12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:16.444910 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7391afd-82e1-4cf8-b63f-416cfe3da12e admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.445239 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7391afd-82e1-4cf8-b63f-416cfe3da12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.445517 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7391afd-82e1-4cf8-b63f-416cfe3da12e admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.446428 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7391afd-82e1-4cf8-b63f-416cfe3da12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:16.448258 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7391afd-82e1-4cf8-b63f-416cfe3da12e admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:16.457662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1377/2766] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:16 2022] PUT /identity/v3/projects/5cdf1fff648c440eb3b69dc25b5d8dc6/users/782acde280984ec6a01d788c95754f35/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:16.461394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1390/2767] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:32:16.465900 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef86bb6-206a-4c28-8107-9350c3a2f30c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:16.467960 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed5545-ed5c-448d-8331-3dbe1ae4841c None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.468241 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed5545-ed5c-448d-8331-3dbe1ae4841c None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.468510 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed5545-ed5c-448d-8331-3dbe1ae4841c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.510391 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-4fed5545-ed5c-448d-8331-3dbe1ae4841c None None] MFA Rules not processed for user `782acde280984ec6a01d788c95754f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:16.511609 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4fed5545-ed5c-448d-8331-3dbe1ae4841c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:16.511978 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:16.559884 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef86bb6-206a-4c28-8107-9350c3a2f30c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:16.564991 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cf928-54a7-481c-b5ee-f38100cc4033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:16.565951 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cf928-54a7-481c-b5ee-f38100cc4033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.566241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cf928-54a7-481c-b5ee-f38100cc4033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.566588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cf928-54a7-481c-b5ee-f38100cc4033 admin admin] PATH_INFO: `/v3/projects/d95af0d3b2b9487ea3a1bef307e78369/users/493ea99d4e1c414f8275aa196f8d8728/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.585301 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cf928-54a7-481c-b5ee-f38100cc4033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d95af0d3b2b9487ea3a1bef307e78369, user_id=493ea99d4e1c414f8275aa196f8d8728, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:16.587970 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1391/2768] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:16.589368 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cf928-54a7-481c-b5ee-f38100cc4033 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:16.601146 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed5545-ed5c-448d-8331-3dbe1ae4841c None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:16.602828 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4fed5545-ed5c-448d-8331-3dbe1ae4841c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:16.606545 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1378/2769] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:16 2022] PUT /identity/v3/projects/d95af0d3b2b9487ea3a1bef307e78369/users/493ea99d4e1c414f8275aa196f8d8728/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:16.611478 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cf928-54a7-481c-b5ee-f38100cc4033 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:16.657879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:16.685485 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed5545-ed5c-448d-8331-3dbe1ae4841c None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:16.693598 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cf928-54a7-481c-b5ee-f38100cc4033 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:16.698371 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20150016-37fc-4373-9f01-51325d4bf953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:16.699001 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20150016-37fc-4373-9f01-51325d4bf953 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.699285 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20150016-37fc-4373-9f01-51325d4bf953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.699559 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20150016-37fc-4373-9f01-51325d4bf953 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.700251 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20150016-37fc-4373-9f01-51325d4bf953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:16.702479 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20150016-37fc-4373-9f01-51325d4bf953 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:16.714024 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1379/2770] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:16.718237 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20150016-37fc-4373-9f01-51325d4bf953 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:16.728620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20150016-37fc-4373-9f01-51325d4bf953 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:16.737026 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56b80b-9a65-4a04-8ecf-0d1b55c5580c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:16.737728 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56b80b-9a65-4a04-8ecf-0d1b55c5580c admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.738037 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56b80b-9a65-4a04-8ecf-0d1b55c5580c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.738313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56b80b-9a65-4a04-8ecf-0d1b55c5580c admin admin] PATH_INFO: `/v3/projects/d95af0d3b2b9487ea3a1bef307e78369/users/493ea99d4e1c414f8275aa196f8d8728/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.738948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc9328-9f70-4261-8f8e-04b65ac57776 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:16.739714 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc9328-9f70-4261-8f8e-04b65ac57776 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.740044 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc9328-9f70-4261-8f8e-04b65ac57776 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.740293 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc9328-9f70-4261-8f8e-04b65ac57776 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.740516 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56b80b-9a65-4a04-8ecf-0d1b55c5580c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d95af0d3b2b9487ea3a1bef307e78369, user_id=493ea99d4e1c414f8275aa196f8d8728, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:16.743271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56b80b-9a65-4a04-8ecf-0d1b55c5580c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:16.754343 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1380/2771] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:16 2022] PUT /identity/v3/projects/d95af0d3b2b9487ea3a1bef307e78369/users/493ea99d4e1c414f8275aa196f8d8728/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:16.765407 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91e666-bc7c-47e7-a9dd-a42a823189c7 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.765694 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91e666-bc7c-47e7-a9dd-a42a823189c7 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.766336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91e666-bc7c-47e7-a9dd-a42a823189c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.799363 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-7d91e666-bc7c-47e7-a9dd-a42a823189c7 None None] MFA Rules not processed for user `493ea99d4e1c414f8275aa196f8d8728`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:16.800986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7d91e666-bc7c-47e7-a9dd-a42a823189c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:16.807952 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc9328-9f70-4261-8f8e-04b65ac57776 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:16.809497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc9328-9f70-4261-8f8e-04b65ac57776 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:16.821467 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1392/2772] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:32:16.877684 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1381/2773] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:16.889642 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc9328-9f70-4261-8f8e-04b65ac57776 service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:16.891505 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-d0fc9328-9f70-4261-8f8e-04b65ac57776 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:16.902538 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc9328-9f70-4261-8f8e-04b65ac57776 service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:16.960931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12533a-75a6-43c0-9371-d7c871eb9254 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:16.961570 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12533a-75a6-43c0-9371-d7c871eb9254 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:16.961888 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12533a-75a6-43c0-9371-d7c871eb9254 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:16.962165 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12533a-75a6-43c0-9371-d7c871eb9254 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:16.972662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12533a-75a6-43c0-9371-d7c871eb9254 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:16.974543 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12533a-75a6-43c0-9371-d7c871eb9254 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:16.990615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1393/2774] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:17.435599 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91e666-bc7c-47e7-a9dd-a42a823189c7 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:17.462914 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:17.521263 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91e666-bc7c-47e7-a9dd-a42a823189c7 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:17.527323 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22f431-26cb-4e69-a976-de12bb123ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:17.528034 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22f431-26cb-4e69-a976-de12bb123ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:17.528314 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22f431-26cb-4e69-a976-de12bb123ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:17.528575 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22f431-26cb-4e69-a976-de12bb123ab5 admin admin] PATH_INFO: `/v3/users/dc5b1f13b8234e04b97236e66e2b21d9` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:17.530013 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22f431-26cb-4e69-a976-de12bb123ab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc5b1f13b8234e04b97236e66e2b21d9)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:17.532153 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22f431-26cb-4e69-a976-de12bb123ab5 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:17.586258 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1b22f431-26cb-4e69-a976-de12bb123ab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc5b1f13b8234e04b97236e66e2b21d9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:17.586921 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1b22f431-26cb-4e69-a976-de12bb123ab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc5b1f13b8234e04b97236e66e2b21d9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:17.599925 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1b22f431-26cb-4e69-a976-de12bb123ab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc5b1f13b8234e04b97236e66e2b21d9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:17.607749 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1b22f431-26cb-4e69-a976-de12bb123ab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc5b1f13b8234e04b97236e66e2b21d9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:17.616242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-1b22f431-26cb-4e69-a976-de12bb123ab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc5b1f13b8234e04b97236e66e2b21d9'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:17.622439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1382/2775] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:17 2022] DELETE /identity/v3/users/dc5b1f13b8234e04b97236e66e2b21d9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:17.627292 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f52ca-e272-4dcd-9f27-c794da676fc8 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:17.627787 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f52ca-e272-4dcd-9f27-c794da676fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:17.627787 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f52ca-e272-4dcd-9f27-c794da676fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:17.647710 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-cd2f52ca-e272-4dcd-9f27-c794da676fc8 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:17.649049 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-cd2f52ca-e272-4dcd-9f27-c794da676fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:17.697126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:17.719274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1394/2776] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:17.730465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22f431-26cb-4e69-a976-de12bb123ab5 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:17.732185 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1b22f431-26cb-4e69-a976-de12bb123ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:17.797932 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22f431-26cb-4e69-a976-de12bb123ab5 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:17.802182 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74426532-7363-4073-85f9-79026fcad5dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:17.802827 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74426532-7363-4073-85f9-79026fcad5dc service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:17.803109 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74426532-7363-4073-85f9-79026fcad5dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:17.803385 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74426532-7363-4073-85f9-79026fcad5dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:17.813766 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74426532-7363-4073-85f9-79026fcad5dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:17.815309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74426532-7363-4073-85f9-79026fcad5dc service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:17.825804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1383/2777] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:18.166649 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f52ca-e272-4dcd-9f27-c794da676fc8 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:18.168182 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-cd2f52ca-e272-4dcd-9f27-c794da676fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:18.182480 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f52ca-e272-4dcd-9f27-c794da676fc8 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:18.187805 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e9f7a-623a-4cd8-9606-335369def205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:18.188542 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e9f7a-623a-4cd8-9606-335369def205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:18.188874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e9f7a-623a-4cd8-9606-335369def205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:18.189207 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e9f7a-623a-4cd8-9606-335369def205 admin admin] PATH_INFO: `/v3/projects/ecd39538ef64401e831ff44fcae96962` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:18.190590 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e9f7a-623a-4cd8-9606-335369def205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecd39538ef64401e831ff44fcae96962)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:18.192500 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e9f7a-623a-4cd8-9606-335369def205 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:18.245166 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-e62e9f7a-623a-4cd8-9606-335369def205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecd39538ef64401e831ff44fcae96962', 'request_id': 'req-e62e9f7a-623a-4cd8-9606-335369def205'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:18.253126 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-e62e9f7a-623a-4cd8-9606-335369def205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecd39538ef64401e831ff44fcae96962', 'request_id': 'req-e62e9f7a-623a-4cd8-9606-335369def205'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:18.260053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-e62e9f7a-623a-4cd8-9606-335369def205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecd39538ef64401e831ff44fcae96962', 'request_id': 'req-e62e9f7a-623a-4cd8-9606-335369def205'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:18.271673 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-e62e9f7a-623a-4cd8-9606-335369def205 admin admin] The token cache is being invalidate because project ecd39538ef64401e831ff44fcae96962 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:18.271958 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-e62e9f7a-623a-4cd8-9606-335369def205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:18.273056 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-e62e9f7a-623a-4cd8-9606-335369def205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:18.274655 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1395/2778] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:18 2022] DELETE /identity/v3/projects/ecd39538ef64401e831ff44fcae96962 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:18.278842 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74426532-7363-4073-85f9-79026fcad5dc service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:18.289401 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74426532-7363-4073-85f9-79026fcad5dc service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:18.337213 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:18.363045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9404d-8344-4148-8556-6603f69e70ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:18.363237 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9404d-8344-4148-8556-6603f69e70ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:18.363519 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9404d-8344-4148-8556-6603f69e70ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:18.363799 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9404d-8344-4148-8556-6603f69e70ae admin admin] PATH_INFO: `/v3/users/493ea99d4e1c414f8275aa196f8d8728` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:18.365045 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9404d-8344-4148-8556-6603f69e70ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=493ea99d4e1c414f8275aa196f8d8728)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:18.367125 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9404d-8344-4148-8556-6603f69e70ae admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:18.417741 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b6c9404d-8344-4148-8556-6603f69e70ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '493ea99d4e1c414f8275aa196f8d8728'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:18.418422 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b6c9404d-8344-4148-8556-6603f69e70ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '493ea99d4e1c414f8275aa196f8d8728'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:18.430988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b6c9404d-8344-4148-8556-6603f69e70ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '493ea99d4e1c414f8275aa196f8d8728'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:18.440544 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b6c9404d-8344-4148-8556-6603f69e70ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '493ea99d4e1c414f8275aa196f8d8728'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:18.448264 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b6c9404d-8344-4148-8556-6603f69e70ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '493ea99d4e1c414f8275aa196f8d8728'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:18.458115 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1384/2779] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:18 2022] DELETE /identity/v3/users/493ea99d4e1c414f8275aa196f8d8728 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:18.760344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e9f7a-623a-4cd8-9606-335369def205 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:18.761912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e62e9f7a-623a-4cd8-9606-335369def205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:18.776999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e9f7a-623a-4cd8-9606-335369def205 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:18.844278 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a99e24-3993-4533-a3c9-ce36ec328b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:18.844937 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a99e24-3993-4533-a3c9-ce36ec328b32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:18.845224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a99e24-3993-4533-a3c9-ce36ec328b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:18.845500 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a99e24-3993-4533-a3c9-ce36ec328b32 admin admin] PATH_INFO: `/v3/projects/d95af0d3b2b9487ea3a1bef307e78369` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:18.846771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a99e24-3993-4533-a3c9-ce36ec328b32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d95af0d3b2b9487ea3a1bef307e78369)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:18.848706 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a99e24-3993-4533-a3c9-ce36ec328b32 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:18.907793 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-54a99e24-3993-4533-a3c9-ce36ec328b32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd95af0d3b2b9487ea3a1bef307e78369', 'request_id': 'req-54a99e24-3993-4533-a3c9-ce36ec328b32'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:18.921122 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-54a99e24-3993-4533-a3c9-ce36ec328b32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd95af0d3b2b9487ea3a1bef307e78369', 'request_id': 'req-54a99e24-3993-4533-a3c9-ce36ec328b32'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:18.928980 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-54a99e24-3993-4533-a3c9-ce36ec328b32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd95af0d3b2b9487ea3a1bef307e78369', 'request_id': 'req-54a99e24-3993-4533-a3c9-ce36ec328b32'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:18.936826 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:18.941591 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-54a99e24-3993-4533-a3c9-ce36ec328b32 admin admin] The token cache is being invalidate because project d95af0d3b2b9487ea3a1bef307e78369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:18.941803 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-54a99e24-3993-4533-a3c9-ce36ec328b32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:18.942548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-54a99e24-3993-4533-a3c9-ce36ec328b32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:18.944840 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1396/2780] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:18 2022] DELETE /identity/v3/projects/d95af0d3b2b9487ea3a1bef307e78369 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:18.950137 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2569ec2-d8fc-4120-b1c0-1fe5bc617645 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:18.950537 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2569ec2-d8fc-4120-b1c0-1fe5bc617645 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:18.950948 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2569ec2-d8fc-4120-b1c0-1fe5bc617645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:18.971721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-f2569ec2-d8fc-4120-b1c0-1fe5bc617645 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:18.972954 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f2569ec2-d8fc-4120-b1c0-1fe5bc617645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:19.039863 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1385/2781] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:19.049320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74955bcb-4649-4e70-baf8-7d8ce1608ce3 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:19.049569 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74955bcb-4649-4e70-baf8-7d8ce1608ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:19.049896 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74955bcb-4649-4e70-baf8-7d8ce1608ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:19.070100 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-74955bcb-4649-4e70-baf8-7d8ce1608ce3 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:19.071322 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-74955bcb-4649-4e70-baf8-7d8ce1608ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:19.079733 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1397/2782] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:19.084276 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2569ec2-d8fc-4120-b1c0-1fe5bc617645 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:19.094591 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2569ec2-d8fc-4120-b1c0-1fe5bc617645 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:19.099434 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b8d3c-3ad9-4197-921b-6fe0f5abe777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:19.100068 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b8d3c-3ad9-4197-921b-6fe0f5abe777 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:19.100347 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b8d3c-3ad9-4197-921b-6fe0f5abe777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:19.100618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b8d3c-3ad9-4197-921b-6fe0f5abe777 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:19.101341 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b8d3c-3ad9-4197-921b-6fe0f5abe777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:32:19.101644 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b8d3c-3ad9-4197-921b-6fe0f5abe777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:19.103295 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b8d3c-3ad9-4197-921b-6fe0f5abe777 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:19.116767 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1386/2783] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:32:19.121372 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74955bcb-4649-4e70-baf8-7d8ce1608ce3 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:19.131732 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74955bcb-4649-4e70-baf8-7d8ce1608ce3 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:19.136979 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94659a-ff39-47b8-b6df-f91dbc675924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:19.137608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94659a-ff39-47b8-b6df-f91dbc675924 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:19.137911 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94659a-ff39-47b8-b6df-f91dbc675924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:19.138176 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94659a-ff39-47b8-b6df-f91dbc675924 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:19.138839 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94659a-ff39-47b8-b6df-f91dbc675924 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:19.140742 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94659a-ff39-47b8-b6df-f91dbc675924 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:19.171380 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1398/2784] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:32:19 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:19.176262 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b8d3c-3ad9-4197-921b-6fe0f5abe777 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:19.187030 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b8d3c-3ad9-4197-921b-6fe0f5abe777 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:19.250946 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:19.257587 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781ff06-56c2-44ee-aedc-5bac8ef296ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:19.261824 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781ff06-56c2-44ee-aedc-5bac8ef296ee admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:19.262201 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781ff06-56c2-44ee-aedc-5bac8ef296ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:19.262529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781ff06-56c2-44ee-aedc-5bac8ef296ee admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:19.263271 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781ff06-56c2-44ee-aedc-5bac8ef296ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:19.265329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781ff06-56c2-44ee-aedc-5bac8ef296ee admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:19.288636 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1387/2785] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:32:19 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:19.293201 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94659a-ff39-47b8-b6df-f91dbc675924 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:19.304262 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94659a-ff39-47b8-b6df-f91dbc675924 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:19.309645 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76687b86-5b74-4bb1-a32d-b9eb07e31ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:19.310318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76687b86-5b74-4bb1-a32d-b9eb07e31ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:19.310602 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76687b86-5b74-4bb1-a32d-b9eb07e31ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:19.310872 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76687b86-5b74-4bb1-a32d-b9eb07e31ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:19.311587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76687b86-5b74-4bb1-a32d-b9eb07e31ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:19.313188 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76687b86-5b74-4bb1-a32d-b9eb07e31ce2 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:19.324997 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1399/2786] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:32:19.329791 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781ff06-56c2-44ee-aedc-5bac8ef296ee admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:19.341075 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781ff06-56c2-44ee-aedc-5bac8ef296ee admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:19.346368 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822135ef-a870-439d-a6ab-ef9b6dad9a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:19.347049 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822135ef-a870-439d-a6ab-ef9b6dad9a72 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:19.347340 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822135ef-a870-439d-a6ab-ef9b6dad9a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:19.347610 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822135ef-a870-439d-a6ab-ef9b6dad9a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:19.348356 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822135ef-a870-439d-a6ab-ef9b6dad9a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:19.350313 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822135ef-a870-439d-a6ab-ef9b6dad9a72 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:19.362663 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1388/2787] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:19.367504 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76687b86-5b74-4bb1-a32d-b9eb07e31ce2 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:19.379273 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76687b86-5b74-4bb1-a32d-b9eb07e31ce2 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:19.384972 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d756696-f9fc-459d-a6f3-dea948ce5b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:19.385662 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d756696-f9fc-459d-a6f3-dea948ce5b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:19.385972 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d756696-f9fc-459d-a6f3-dea948ce5b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:19.386247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d756696-f9fc-459d-a6f3-dea948ce5b57 admin admin] PATH_INFO: `/v3/projects/7d154ed837c3445691b7baa427dcc89e/users/84d899fbb60a416d9fc4136950edf59b/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:19.403008 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d756696-f9fc-459d-a6f3-dea948ce5b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d154ed837c3445691b7baa427dcc89e, user_id=84d899fbb60a416d9fc4136950edf59b, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:19.405813 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d756696-f9fc-459d-a6f3-dea948ce5b57 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:19.421876 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1400/2788] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:19 2022] PUT /identity/v3/projects/7d154ed837c3445691b7baa427dcc89e/users/84d899fbb60a416d9fc4136950edf59b/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:19.433055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455881d8-a916-45f6-93f9-3c42c3a70825 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:19.433338 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455881d8-a916-45f6-93f9-3c42c3a70825 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:19.433613 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455881d8-a916-45f6-93f9-3c42c3a70825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:19.465998 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-455881d8-a916-45f6-93f9-3c42c3a70825 None None] MFA Rules not processed for user `84d899fbb60a416d9fc4136950edf59b`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:19.467226 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-455881d8-a916-45f6-93f9-3c42c3a70825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:19.544593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1389/2789] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:19.556457 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d756696-f9fc-459d-a6f3-dea948ce5b57 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:19.558429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-3d756696-f9fc-459d-a6f3-dea948ce5b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:19.621695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d756696-f9fc-459d-a6f3-dea948ce5b57 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:19.626688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2150e42-d6c3-4e66-b78a-e351c226decf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f696dfe4b34c4aa19104b5f7ee6e70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:19.626688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2150e42-d6c3-4e66-b78a-e351c226decf service glance] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:19.626688 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2150e42-d6c3-4e66-b78a-e351c226decf service glance] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:19.627133 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2150e42-d6c3-4e66-b78a-e351c226decf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:19.636088 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2150e42-d6c3-4e66-b78a-e351c226decf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:19.637511 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2150e42-d6c3-4e66-b78a-e351c226decf service glance] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:19.646999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1401/2790] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:23.626013 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455881d8-a916-45f6-93f9-3c42c3a70825 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:23.627548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-455881d8-a916-45f6-93f9-3c42c3a70825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:23.703546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455881d8-a916-45f6-93f9-3c42c3a70825 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:23.708716 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17e9d2-cb8d-443a-9a5f-f1f9dbf48fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:23.709457 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17e9d2-cb8d-443a-9a5f-f1f9dbf48fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:23.709809 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17e9d2-cb8d-443a-9a5f-f1f9dbf48fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:23.710287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17e9d2-cb8d-443a-9a5f-f1f9dbf48fee admin admin] PATH_INFO: `/v3/users/240c6d10ab184f98a3f435e769e08146` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:23.711575 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e17e9d2-cb8d-443a-9a5f-f1f9dbf48fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=240c6d10ab184f98a3f435e769e08146)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:23.713825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e17e9d2-cb8d-443a-9a5f-f1f9dbf48fee admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:23.766902 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7e17e9d2-cb8d-443a-9a5f-f1f9dbf48fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '240c6d10ab184f98a3f435e769e08146'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:23.767596 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7e17e9d2-cb8d-443a-9a5f-f1f9dbf48fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '240c6d10ab184f98a3f435e769e08146'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:23.778590 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:23.785990 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7e17e9d2-cb8d-443a-9a5f-f1f9dbf48fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '240c6d10ab184f98a3f435e769e08146'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:23.795778 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7e17e9d2-cb8d-443a-9a5f-f1f9dbf48fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '240c6d10ab184f98a3f435e769e08146'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:23.804265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-7e17e9d2-cb8d-443a-9a5f-f1f9dbf48fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '240c6d10ab184f98a3f435e769e08146'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:23.810811 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1390/2791] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:23 2022] DELETE /identity/v3/users/240c6d10ab184f98a3f435e769e08146 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:23.816084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e3e96-9784-4c18-9bdf-114388fa63f8 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:23.816639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e3e96-9784-4c18-9bdf-114388fa63f8 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:23.816639 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e3e96-9784-4c18-9bdf-114388fa63f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:23.839213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-572e3e96-9784-4c18-9bdf-114388fa63f8 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:23.840451 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-572e3e96-9784-4c18-9bdf-114388fa63f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:23.906033 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1402/2792] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:23.916274 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17e9d2-cb8d-443a-9a5f-f1f9dbf48fee admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:23.918025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-7e17e9d2-cb8d-443a-9a5f-f1f9dbf48fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:23.986635 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17e9d2-cb8d-443a-9a5f-f1f9dbf48fee admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:23.991356 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddcd84-6208-4120-a5d0-2fa4fe935431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:23.991994 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddcd84-6208-4120-a5d0-2fa4fe935431 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:23.992270 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddcd84-6208-4120-a5d0-2fa4fe935431 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:23.992535 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddcd84-6208-4120-a5d0-2fa4fe935431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:24.003248 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ddcd84-6208-4120-a5d0-2fa4fe935431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:24.004714 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ddcd84-6208-4120-a5d0-2fa4fe935431 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:24.016027 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1391/2793] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:24.305379 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e3e96-9784-4c18-9bdf-114388fa63f8 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:24.307038 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-572e3e96-9784-4c18-9bdf-114388fa63f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:24.333750 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e3e96-9784-4c18-9bdf-114388fa63f8 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:24.340692 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801ced6-c898-4e82-aea7-0d0a28cba009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:24.341486 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801ced6-c898-4e82-aea7-0d0a28cba009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:24.341847 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801ced6-c898-4e82-aea7-0d0a28cba009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:24.342598 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801ced6-c898-4e82-aea7-0d0a28cba009 admin admin] PATH_INFO: `/v3/projects/bc3951cdb6f246aaba44f91f0813bde4` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:24.349243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801ced6-c898-4e82-aea7-0d0a28cba009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc3951cdb6f246aaba44f91f0813bde4)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:24.349243 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801ced6-c898-4e82-aea7-0d0a28cba009 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:24.407267 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:24.418444 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8801ced6-c898-4e82-aea7-0d0a28cba009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc3951cdb6f246aaba44f91f0813bde4', 'request_id': 'req-8801ced6-c898-4e82-aea7-0d0a28cba009'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:24.427733 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8801ced6-c898-4e82-aea7-0d0a28cba009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc3951cdb6f246aaba44f91f0813bde4', 'request_id': 'req-8801ced6-c898-4e82-aea7-0d0a28cba009'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:24.435917 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8801ced6-c898-4e82-aea7-0d0a28cba009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc3951cdb6f246aaba44f91f0813bde4', 'request_id': 'req-8801ced6-c898-4e82-aea7-0d0a28cba009'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:24.448392 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8801ced6-c898-4e82-aea7-0d0a28cba009 admin admin] The token cache is being invalidate because project bc3951cdb6f246aaba44f91f0813bde4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:24.448617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8801ced6-c898-4e82-aea7-0d0a28cba009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:24.449128 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-8801ced6-c898-4e82-aea7-0d0a28cba009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:24.450918 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1403/2794] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:24 2022] DELETE /identity/v3/projects/bc3951cdb6f246aaba44f91f0813bde4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:24.456797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddcd84-6208-4120-a5d0-2fa4fe935431 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:24.467593 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddcd84-6208-4120-a5d0-2fa4fe935431 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:24.542025 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e554ada-888f-40c1-bbed-8dce3596e6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:24.542761 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e554ada-888f-40c1-bbed-8dce3596e6b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:24.543022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e554ada-888f-40c1-bbed-8dce3596e6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:24.543493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e554ada-888f-40c1-bbed-8dce3596e6b0 admin admin] PATH_INFO: `/v3/users/b37e6a1f1a09401cb71cbea535b56641` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:24.545239 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e554ada-888f-40c1-bbed-8dce3596e6b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b37e6a1f1a09401cb71cbea535b56641)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:24.547324 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e554ada-888f-40c1-bbed-8dce3596e6b0 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:24.629567 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8e554ada-888f-40c1-bbed-8dce3596e6b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b37e6a1f1a09401cb71cbea535b56641'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:24.630885 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8e554ada-888f-40c1-bbed-8dce3596e6b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b37e6a1f1a09401cb71cbea535b56641'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:24.650721 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8e554ada-888f-40c1-bbed-8dce3596e6b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b37e6a1f1a09401cb71cbea535b56641'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:24.660254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8e554ada-888f-40c1-bbed-8dce3596e6b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b37e6a1f1a09401cb71cbea535b56641'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:24.672635 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8e554ada-888f-40c1-bbed-8dce3596e6b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b37e6a1f1a09401cb71cbea535b56641'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:24.674929 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:24.683015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1392/2795] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:24 2022] DELETE /identity/v3/users/b37e6a1f1a09401cb71cbea535b56641 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:25.013532 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801ced6-c898-4e82-aea7-0d0a28cba009 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:25.015437 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8801ced6-c898-4e82-aea7-0d0a28cba009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:25.031427 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801ced6-c898-4e82-aea7-0d0a28cba009 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:25.101181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f3ba1-9c06-4c35-9427-3517fed64ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:25.101802 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f3ba1-9c06-4c35-9427-3517fed64ebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:25.102084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f3ba1-9c06-4c35-9427-3517fed64ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:25.102446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f3ba1-9c06-4c35-9427-3517fed64ebc admin admin] PATH_INFO: `/v3/projects/76c96cc1e5e9491aa0309ffad71ad326` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:25.103753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f3ba1-9c06-4c35-9427-3517fed64ebc admin admin] RBAC: Authorizing `identity:delete_project(project_id=76c96cc1e5e9491aa0309ffad71ad326)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:25.105676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f3ba1-9c06-4c35-9427-3517fed64ebc admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:25.161290 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ab8f3ba1-9c06-4c35-9427-3517fed64ebc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76c96cc1e5e9491aa0309ffad71ad326', 'request_id': 'req-ab8f3ba1-9c06-4c35-9427-3517fed64ebc'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:25.171772 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ab8f3ba1-9c06-4c35-9427-3517fed64ebc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76c96cc1e5e9491aa0309ffad71ad326', 'request_id': 'req-ab8f3ba1-9c06-4c35-9427-3517fed64ebc'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:25.179046 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ab8f3ba1-9c06-4c35-9427-3517fed64ebc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76c96cc1e5e9491aa0309ffad71ad326', 'request_id': 'req-ab8f3ba1-9c06-4c35-9427-3517fed64ebc'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:25.192343 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ab8f3ba1-9c06-4c35-9427-3517fed64ebc admin admin] The token cache is being invalidate because project 76c96cc1e5e9491aa0309ffad71ad326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:25.192714 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ab8f3ba1-9c06-4c35-9427-3517fed64ebc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:25.193429 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ab8f3ba1-9c06-4c35-9427-3517fed64ebc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:25.195106 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1404/2796] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:25 2022] DELETE /identity/v3/projects/76c96cc1e5e9491aa0309ffad71ad326 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:25.201378 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51ed87-c4c4-49ee-9ca4-5afb1b00e0fa None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:25.201662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51ed87-c4c4-49ee-9ca4-5afb1b00e0fa None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:25.201954 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51ed87-c4c4-49ee-9ca4-5afb1b00e0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:25.224722 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-2f51ed87-c4c4-49ee-9ca4-5afb1b00e0fa None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:25.226009 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-2f51ed87-c4c4-49ee-9ca4-5afb1b00e0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:25.300357 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1393/2797] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:25.309818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5aaaf-29f3-4313-af0c-464b2768a043 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:25.310092 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5aaaf-29f3-4313-af0c-464b2768a043 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:25.310382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5aaaf-29f3-4313-af0c-464b2768a043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:25.332671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-05f5aaaf-29f3-4313-af0c-464b2768a043 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:25.333982 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-05f5aaaf-29f3-4313-af0c-464b2768a043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:25.342900 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1405/2798] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:25.349116 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51ed87-c4c4-49ee-9ca4-5afb1b00e0fa None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:25.360875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51ed87-c4c4-49ee-9ca4-5afb1b00e0fa None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:25.365702 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e50ec-5b77-4703-98d4-e8f91f8a3fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:25.366450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e50ec-5b77-4703-98d4-e8f91f8a3fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:25.366732 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e50ec-5b77-4703-98d4-e8f91f8a3fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:25.367002 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e50ec-5b77-4703-98d4-e8f91f8a3fc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:25.367709 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e50ec-5b77-4703-98d4-e8f91f8a3fc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80291) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:32:25.368055 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e50ec-5b77-4703-98d4-e8f91f8a3fc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:25.369673 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e50ec-5b77-4703-98d4-e8f91f8a3fc1 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:25.384195 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1394/2799] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:32:25.389404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5aaaf-29f3-4313-af0c-464b2768a043 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:25.399764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5aaaf-29f3-4313-af0c-464b2768a043 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:25.404767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe0634-783c-4aab-be38-aa87faa9d532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:25.405540 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe0634-783c-4aab-be38-aa87faa9d532 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:25.405709 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe0634-783c-4aab-be38-aa87faa9d532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:25.406062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe0634-783c-4aab-be38-aa87faa9d532 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:25.406672 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe0634-783c-4aab-be38-aa87faa9d532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:25.408574 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe0634-783c-4aab-be38-aa87faa9d532 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:25.440692 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1406/2800] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:32:25 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:25.445147 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e50ec-5b77-4703-98d4-e8f91f8a3fc1 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:25.462252 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e50ec-5b77-4703-98d4-e8f91f8a3fc1 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:25.524882 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2816a-b884-4028-9af3-dce0818bb25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:25.525522 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2816a-b884-4028-9af3-dce0818bb25a admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:25.525867 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2816a-b884-4028-9af3-dce0818bb25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:25.526142 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2816a-b884-4028-9af3-dce0818bb25a admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:25.526829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec2816a-b884-4028-9af3-dce0818bb25a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:25.528777 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec2816a-b884-4028-9af3-dce0818bb25a admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:25.553634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1395/2801] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:32:25 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:25.558057 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe0634-783c-4aab-be38-aa87faa9d532 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:25.569695 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe0634-783c-4aab-be38-aa87faa9d532 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:25.574705 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cc198-8a38-4d32-8ad8-16b2700b63e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:25.575319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cc198-8a38-4d32-8ad8-16b2700b63e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:25.575599 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cc198-8a38-4d32-8ad8-16b2700b63e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:25.575871 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64cc198-8a38-4d32-8ad8-16b2700b63e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:25.576616 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64cc198-8a38-4d32-8ad8-16b2700b63e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:25.578682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64cc198-8a38-4d32-8ad8-16b2700b63e3 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:25.591275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1407/2802] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:25.595376 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2816a-b884-4028-9af3-dce0818bb25a admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:25.606499 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2816a-b884-4028-9af3-dce0818bb25a admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:25.612589 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b0438-a911-4d04-983b-9ecc4b3b6c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:25.613254 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b0438-a911-4d04-983b-9ecc4b3b6c0c admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:25.613662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b0438-a911-4d04-983b-9ecc4b3b6c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:25.614026 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b0438-a911-4d04-983b-9ecc4b3b6c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:25.614756 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b0438-a911-4d04-983b-9ecc4b3b6c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:25.616427 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b0438-a911-4d04-983b-9ecc4b3b6c0c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:25.627931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1396/2803] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:25.632417 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cc198-8a38-4d32-8ad8-16b2700b63e3 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:25.643591 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64cc198-8a38-4d32-8ad8-16b2700b63e3 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:25.648693 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88d9d9-33f0-4f68-a4ad-cd82d49e614c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:25.649452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88d9d9-33f0-4f68-a4ad-cd82d49e614c admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:25.649733 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88d9d9-33f0-4f68-a4ad-cd82d49e614c admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:25.650053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88d9d9-33f0-4f68-a4ad-cd82d49e614c admin admin] PATH_INFO: `/v3/projects/131d052729d743b0a376393c8bc17e6c/users/e170ce50fe3f4c1e8fdb8fc4bef0001e/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:25.663100 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:25.668152 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88d9d9-33f0-4f68-a4ad-cd82d49e614c admin admin] RBAC: Authorizing `identity:create_grant(project_id=131d052729d743b0a376393c8bc17e6c, user_id=e170ce50fe3f4c1e8fdb8fc4bef0001e, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:25.671670 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88d9d9-33f0-4f68-a4ad-cd82d49e614c admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:25.687541 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1408/2804] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:25 2022] PUT /identity/v3/projects/131d052729d743b0a376393c8bc17e6c/users/e170ce50fe3f4c1e8fdb8fc4bef0001e/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:25.698218 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f182013-630d-43ed-81f9-9fee84dd3896 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:25.698833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f182013-630d-43ed-81f9-9fee84dd3896 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:25.698833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f182013-630d-43ed-81f9-9fee84dd3896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:25.731713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-1f182013-630d-43ed-81f9-9fee84dd3896 None None] MFA Rules not processed for user `e170ce50fe3f4c1e8fdb8fc4bef0001e`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:25.733022 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-1f182013-630d-43ed-81f9-9fee84dd3896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:25.801346 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:25.813756 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1397/2805] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:25.818508 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88d9d9-33f0-4f68-a4ad-cd82d49e614c admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:25.900344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88d9d9-33f0-4f68-a4ad-cd82d49e614c admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:25.905401 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806ba0a-1a64-4692-9c7b-7a37ea08d5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:25.906204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806ba0a-1a64-4692-9c7b-7a37ea08d5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:25.906521 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806ba0a-1a64-4692-9c7b-7a37ea08d5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:25.906744 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806ba0a-1a64-4692-9c7b-7a37ea08d5e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:25.907422 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806ba0a-1a64-4692-9c7b-7a37ea08d5e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:25.909341 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806ba0a-1a64-4692-9c7b-7a37ea08d5e7 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:25.941956 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1409/2806] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:32:25 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:25.943986 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f182013-630d-43ed-81f9-9fee84dd3896 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:25.957458 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f182013-630d-43ed-81f9-9fee84dd3896 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:26.022920 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21437ba4-6d46-4094-91a9-c486f879103b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:26.023564 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21437ba4-6d46-4094-91a9-c486f879103b admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:26.023844 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21437ba4-6d46-4094-91a9-c486f879103b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:26.024134 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21437ba4-6d46-4094-91a9-c486f879103b admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:26.024963 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21437ba4-6d46-4094-91a9-c486f879103b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:26.026962 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21437ba4-6d46-4094-91a9-c486f879103b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:26.073757 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1398/2807] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:32:25 2022] POST /identity/v3/users => generated 299 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:26.075362 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806ba0a-1a64-4692-9c7b-7a37ea08d5e7 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:26.089880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806ba0a-1a64-4692-9c7b-7a37ea08d5e7 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:26.095205 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62b117-54f4-4986-8bb5-17248c27d2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:26.095857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62b117-54f4-4986-8bb5-17248c27d2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:26.096119 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62b117-54f4-4986-8bb5-17248c27d2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:26.096497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62b117-54f4-4986-8bb5-17248c27d2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:26.097162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62b117-54f4-4986-8bb5-17248c27d2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:26.098796 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f62b117-54f4-4986-8bb5-17248c27d2c7 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:26.112105 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1410/2808] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:26.116439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21437ba4-6d46-4094-91a9-c486f879103b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:26.127361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21437ba4-6d46-4094-91a9-c486f879103b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:26.132784 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc490b0e-b2e5-4f56-8bdb-a675dbcc6440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:26.133500 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc490b0e-b2e5-4f56-8bdb-a675dbcc6440 admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:26.133963 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc490b0e-b2e5-4f56-8bdb-a675dbcc6440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:26.134259 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc490b0e-b2e5-4f56-8bdb-a675dbcc6440 admin admin] PATH_INFO: `/v3/projects/970f8dfdd3d94be1a71007ac1f1554fd/users/41145195eb0d4b9e931868a5cea2d267/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:26.152847 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc490b0e-b2e5-4f56-8bdb-a675dbcc6440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=970f8dfdd3d94be1a71007ac1f1554fd, user_id=41145195eb0d4b9e931868a5cea2d267, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:26.155546 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc490b0e-b2e5-4f56-8bdb-a675dbcc6440 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:26.173223 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1399/2809] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:26 2022] PUT /identity/v3/projects/970f8dfdd3d94be1a71007ac1f1554fd/users/41145195eb0d4b9e931868a5cea2d267/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:26.180532 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62b117-54f4-4986-8bb5-17248c27d2c7 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:26.192136 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f62b117-54f4-4986-8bb5-17248c27d2c7 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:26.263148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a68fd2-99ec-4bd6-9a3a-b3ae19ad53b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:26.263955 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a68fd2-99ec-4bd6-9a3a-b3ae19ad53b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:26.264303 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a68fd2-99ec-4bd6-9a3a-b3ae19ad53b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:26.264608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a68fd2-99ec-4bd6-9a3a-b3ae19ad53b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:26.265396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a68fd2-99ec-4bd6-9a3a-b3ae19ad53b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:26.267767 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a68fd2-99ec-4bd6-9a3a-b3ae19ad53b7 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:26.280218 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1411/2810] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:26.284781 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc490b0e-b2e5-4f56-8bdb-a675dbcc6440 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:26.296717 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc490b0e-b2e5-4f56-8bdb-a675dbcc6440 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:26.303461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d6d61-5ba3-4628-8c0c-0645902e01ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:26.304353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d6d61-5ba3-4628-8c0c-0645902e01ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:26.304687 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d6d61-5ba3-4628-8c0c-0645902e01ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:26.304957 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d6d61-5ba3-4628-8c0c-0645902e01ab admin admin] PATH_INFO: `/v3/projects/970f8dfdd3d94be1a71007ac1f1554fd/users/41145195eb0d4b9e931868a5cea2d267/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:26.307260 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d6d61-5ba3-4628-8c0c-0645902e01ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=970f8dfdd3d94be1a71007ac1f1554fd, user_id=41145195eb0d4b9e931868a5cea2d267, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:26.309941 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d6d61-5ba3-4628-8c0c-0645902e01ab admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:26.321417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1400/2811] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:26 2022] PUT /identity/v3/projects/970f8dfdd3d94be1a71007ac1f1554fd/users/41145195eb0d4b9e931868a5cea2d267/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:26.335318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc44e8-3518-4f40-bf64-a1efac9f37e7 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:26.335318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc44e8-3518-4f40-bf64-a1efac9f37e7 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:26.335318 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc44e8-3518-4f40-bf64-a1efac9f37e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:26.367463 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-01fc44e8-3518-4f40-bf64-a1efac9f37e7 None None] MFA Rules not processed for user `41145195eb0d4b9e931868a5cea2d267`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:26.368979 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-01fc44e8-3518-4f40-bf64-a1efac9f37e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:26.428458 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:26.454690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1412/2812] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:26.469498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d6d61-5ba3-4628-8c0c-0645902e01ab admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:26.472448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-5c5d6d61-5ba3-4628-8c0c-0645902e01ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:26.552796 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d6d61-5ba3-4628-8c0c-0645902e01ab admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:26.557417 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bde37-bc00-4a73-af9c-f6fbafcd3ad7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:26.559331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bde37-bc00-4a73-af9c-f6fbafcd3ad7 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:26.559331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bde37-bc00-4a73-af9c-f6fbafcd3ad7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:26.559331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bde37-bc00-4a73-af9c-f6fbafcd3ad7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:26.622772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149bde37-bc00-4a73-af9c-f6fbafcd3ad7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:26.624424 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149bde37-bc00-4a73-af9c-f6fbafcd3ad7 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:26.629668 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:26.635875 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1401/2813] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:36.328789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc44e8-3518-4f40-bf64-a1efac9f37e7 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:36.330518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-01fc44e8-3518-4f40-bf64-a1efac9f37e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:36.346212 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc44e8-3518-4f40-bf64-a1efac9f37e7 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:36.350948 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa97666-f566-4ff0-9aaa-e5242deba743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:36.351594 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa97666-f566-4ff0-9aaa-e5242deba743 service cinder] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:36.352137 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa97666-f566-4ff0-9aaa-e5242deba743 service cinder] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:36.352580 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa97666-f566-4ff0-9aaa-e5242deba743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:36.363297 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa97666-f566-4ff0-9aaa-e5242deba743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:36.364784 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa97666-f566-4ff0-9aaa-e5242deba743 service cinder] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:36.375230 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1413/2814] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:40.397035 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bde37-bc00-4a73-af9c-f6fbafcd3ad7 service cinder] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:40.398632 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-149bde37-bc00-4a73-af9c-f6fbafcd3ad7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:40.465454 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bde37-bc00-4a73-af9c-f6fbafcd3ad7 service cinder] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:40.469513 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e86fd-3b10-4ba1-ac68-43dd965eb716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:40.470245 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e86fd-3b10-4ba1-ac68-43dd965eb716 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:40.470617 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e86fd-3b10-4ba1-ac68-43dd965eb716 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:40.471034 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e86fd-3b10-4ba1-ac68-43dd965eb716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:40.474209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d67e86fd-3b10-4ba1-ac68-43dd965eb716 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:40.541682 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e86fd-3b10-4ba1-ac68-43dd965eb716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:40.543281 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e86fd-3b10-4ba1-ac68-43dd965eb716 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:40.552743 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1402/2815] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:41.411905 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa97666-f566-4ff0-9aaa-e5242deba743 service cinder] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:41.583994 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa97666-f566-4ff0-9aaa-e5242deba743 service cinder] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:41.589586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bfca5-5971-495e-a104-1d543b73a3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:41.590617 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bfca5-5971-495e-a104-1d543b73a3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:41.591098 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bfca5-5971-495e-a104-1d543b73a3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:41.591352 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bfca5-5971-495e-a104-1d543b73a3d5 admin admin] PATH_INFO: `/v3/users/e170ce50fe3f4c1e8fdb8fc4bef0001e` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:41.592862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bfca5-5971-495e-a104-1d543b73a3d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e170ce50fe3f4c1e8fdb8fc4bef0001e)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:41.595899 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bfca5-5971-495e-a104-1d543b73a3d5 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:41.723682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-5b6bfca5-5971-495e-a104-1d543b73a3d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e170ce50fe3f4c1e8fdb8fc4bef0001e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:41.744522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-5b6bfca5-5971-495e-a104-1d543b73a3d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e170ce50fe3f4c1e8fdb8fc4bef0001e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:41.744522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-5b6bfca5-5971-495e-a104-1d543b73a3d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e170ce50fe3f4c1e8fdb8fc4bef0001e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:41.760458 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-5b6bfca5-5971-495e-a104-1d543b73a3d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e170ce50fe3f4c1e8fdb8fc4bef0001e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:41.783492 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-5b6bfca5-5971-495e-a104-1d543b73a3d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e170ce50fe3f4c1e8fdb8fc4bef0001e'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:41.794181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1414/2816] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:41 2022] DELETE /identity/v3/users/e170ce50fe3f4c1e8fdb8fc4bef0001e => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:41.799250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a5dd5-022e-4200-9445-6c64cd3fb412 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:41.800064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a5dd5-022e-4200-9445-6c64cd3fb412 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:41.800064 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a5dd5-022e-4200-9445-6c64cd3fb412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:41.825134 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-102a5dd5-022e-4200-9445-6c64cd3fb412 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:41.826675 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-102a5dd5-022e-4200-9445-6c64cd3fb412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:41.945819 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1403/2817] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:41.958436 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bfca5-5971-495e-a104-1d543b73a3d5 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:41.960520 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-5b6bfca5-5971-495e-a104-1d543b73a3d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:42.040977 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bfca5-5971-495e-a104-1d543b73a3d5 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:42.045522 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551cfb4-1077-4217-8f8d-a9efb0967e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:42.046117 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551cfb4-1077-4217-8f8d-a9efb0967e6e service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:42.046553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551cfb4-1077-4217-8f8d-a9efb0967e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:42.046839 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551cfb4-1077-4217-8f8d-a9efb0967e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:42.053857 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:42.059299 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551cfb4-1077-4217-8f8d-a9efb0967e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:42.060874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551cfb4-1077-4217-8f8d-a9efb0967e6e service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:42.073840 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1415/2818] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:32:42.424187 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a5dd5-022e-4200-9445-6c64cd3fb412 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:42.425933 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-102a5dd5-022e-4200-9445-6c64cd3fb412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:42.456558 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a5dd5-022e-4200-9445-6c64cd3fb412 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:42.462514 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ec4cb-7e5b-495a-8b9b-b79345d9363c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:42.463501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ec4cb-7e5b-495a-8b9b-b79345d9363c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:42.463892 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ec4cb-7e5b-495a-8b9b-b79345d9363c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:42.464169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ec4cb-7e5b-495a-8b9b-b79345d9363c admin admin] PATH_INFO: `/v3/projects/131d052729d743b0a376393c8bc17e6c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:42.465577 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ec4cb-7e5b-495a-8b9b-b79345d9363c admin admin] RBAC: Authorizing `identity:delete_project(project_id=131d052729d743b0a376393c8bc17e6c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:42.468111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ec4cb-7e5b-495a-8b9b-b79345d9363c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:42.548992 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-325ec4cb-7e5b-495a-8b9b-b79345d9363c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '131d052729d743b0a376393c8bc17e6c', 'request_id': 'req-325ec4cb-7e5b-495a-8b9b-b79345d9363c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:42.550919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:42.558443 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-325ec4cb-7e5b-495a-8b9b-b79345d9363c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '131d052729d743b0a376393c8bc17e6c', 'request_id': 'req-325ec4cb-7e5b-495a-8b9b-b79345d9363c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:42.573548 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-325ec4cb-7e5b-495a-8b9b-b79345d9363c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '131d052729d743b0a376393c8bc17e6c', 'request_id': 'req-325ec4cb-7e5b-495a-8b9b-b79345d9363c'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:42.590061 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-325ec4cb-7e5b-495a-8b9b-b79345d9363c admin admin] The token cache is being invalidate because project 131d052729d743b0a376393c8bc17e6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:42.590349 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-325ec4cb-7e5b-495a-8b9b-b79345d9363c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:42.591015 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-325ec4cb-7e5b-495a-8b9b-b79345d9363c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:42.592573 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1404/2819] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:42 2022] DELETE /identity/v3/projects/131d052729d743b0a376393c8bc17e6c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:42.597389 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551cfb4-1077-4217-8f8d-a9efb0967e6e service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:42.617198 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551cfb4-1077-4217-8f8d-a9efb0967e6e service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:42.689404 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7f8f7-ca01-4f92-863f-96ed2cf77f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:42.690111 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7f8f7-ca01-4f92-863f-96ed2cf77f83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:42.690384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7f8f7-ca01-4f92-863f-96ed2cf77f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:42.690654 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7f8f7-ca01-4f92-863f-96ed2cf77f83 admin admin] PATH_INFO: `/v3/users/41145195eb0d4b9e931868a5cea2d267` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:42.691870 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7f8f7-ca01-4f92-863f-96ed2cf77f83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41145195eb0d4b9e931868a5cea2d267)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:42.693841 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7f8f7-ca01-4f92-863f-96ed2cf77f83 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:42.786542 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-aac7f8f7-ca01-4f92-863f-96ed2cf77f83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41145195eb0d4b9e931868a5cea2d267'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:42.797296 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-aac7f8f7-ca01-4f92-863f-96ed2cf77f83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41145195eb0d4b9e931868a5cea2d267'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:42.798081 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-aac7f8f7-ca01-4f92-863f-96ed2cf77f83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41145195eb0d4b9e931868a5cea2d267'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:42.810219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-aac7f8f7-ca01-4f92-863f-96ed2cf77f83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41145195eb0d4b9e931868a5cea2d267'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:42.825862 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-aac7f8f7-ca01-4f92-863f-96ed2cf77f83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41145195eb0d4b9e931868a5cea2d267'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:42.834676 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1416/2820] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:42 2022] DELETE /identity/v3/users/41145195eb0d4b9e931868a5cea2d267 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:43.062063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ec4cb-7e5b-495a-8b9b-b79345d9363c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:43.066122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-325ec4cb-7e5b-495a-8b9b-b79345d9363c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:43.085926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ec4cb-7e5b-495a-8b9b-b79345d9363c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:43.173091 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c92403-be72-4810-a0b1-9b4842373408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:43.173742 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c92403-be72-4810-a0b1-9b4842373408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:43.174070 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c92403-be72-4810-a0b1-9b4842373408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:43.174410 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c92403-be72-4810-a0b1-9b4842373408 admin admin] PATH_INFO: `/v3/users/84d899fbb60a416d9fc4136950edf59b` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:43.179432 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c92403-be72-4810-a0b1-9b4842373408 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84d899fbb60a416d9fc4136950edf59b)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:43.183392 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c92403-be72-4810-a0b1-9b4842373408 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:43.184675 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7f8f7-ca01-4f92-863f-96ed2cf77f83 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:43.189474 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-aac7f8f7-ca01-4f92-863f-96ed2cf77f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:43.209024 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7f8f7-ca01-4f92-863f-96ed2cf77f83 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:43.222830 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc029fce-8cd5-4efa-ba59-09c6934445c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:43.223516 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc029fce-8cd5-4efa-ba59-09c6934445c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:43.223794 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc029fce-8cd5-4efa-ba59-09c6934445c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:43.224070 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc029fce-8cd5-4efa-ba59-09c6934445c4 admin admin] PATH_INFO: `/v3/projects/970f8dfdd3d94be1a71007ac1f1554fd` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:43.225384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc029fce-8cd5-4efa-ba59-09c6934445c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=970f8dfdd3d94be1a71007ac1f1554fd)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:43.227456 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc029fce-8cd5-4efa-ba59-09c6934445c4 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:43.288253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d0c92403-be72-4810-a0b1-9b4842373408 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84d899fbb60a416d9fc4136950edf59b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:43.289283 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d0c92403-be72-4810-a0b1-9b4842373408 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84d899fbb60a416d9fc4136950edf59b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:43.302255 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-bc029fce-8cd5-4efa-ba59-09c6934445c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '970f8dfdd3d94be1a71007ac1f1554fd', 'request_id': 'req-bc029fce-8cd5-4efa-ba59-09c6934445c4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:43.308676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d0c92403-be72-4810-a0b1-9b4842373408 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84d899fbb60a416d9fc4136950edf59b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:43.319953 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-bc029fce-8cd5-4efa-ba59-09c6934445c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '970f8dfdd3d94be1a71007ac1f1554fd', 'request_id': 'req-bc029fce-8cd5-4efa-ba59-09c6934445c4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:43.325468 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d0c92403-be72-4810-a0b1-9b4842373408 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84d899fbb60a416d9fc4136950edf59b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:43.329208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-bc029fce-8cd5-4efa-ba59-09c6934445c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '970f8dfdd3d94be1a71007ac1f1554fd', 'request_id': 'req-bc029fce-8cd5-4efa-ba59-09c6934445c4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:43.337083 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:43.340981 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d0c92403-be72-4810-a0b1-9b4842373408 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84d899fbb60a416d9fc4136950edf59b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:43.350427 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1405/2821] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:43 2022] DELETE /identity/v3/users/84d899fbb60a416d9fc4136950edf59b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:43.353608 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-bc029fce-8cd5-4efa-ba59-09c6934445c4 admin admin] The token cache is being invalidate because project 970f8dfdd3d94be1a71007ac1f1554fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:43.354010 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-bc029fce-8cd5-4efa-ba59-09c6934445c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:43.354886 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-bc029fce-8cd5-4efa-ba59-09c6934445c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:43.355459 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836a47e-8382-4024-816c-f3ed304d827e None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:43.355815 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836a47e-8382-4024-816c-f3ed304d827e None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:43.356161 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836a47e-8382-4024-816c-f3ed304d827e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:43.357202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1417/2822] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:43 2022] DELETE /identity/v3/projects/970f8dfdd3d94be1a71007ac1f1554fd => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:43.363084 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003b673-40cc-45fc-80d6-ed38e192fc99 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:43.363449 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003b673-40cc-45fc-80d6-ed38e192fc99 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:43.363638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003b673-40cc-45fc-80d6-ed38e192fc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:43.378782 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-3836a47e-8382-4024-816c-f3ed304d827e None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:43.380250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3836a47e-8382-4024-816c-f3ed304d827e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:43.387296 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-b003b673-40cc-45fc-80d6-ed38e192fc99 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:43.389181 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-b003b673-40cc-45fc-80d6-ed38e192fc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:43.438708 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:43.451872 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1406/2823] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:43.462712 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836a47e-8382-4024-816c-f3ed304d827e None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:43.464140 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3836a47e-8382-4024-816c-f3ed304d827e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:43.468356 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1418/2824] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:43.479144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a9b56-7db9-4c05-b498-7435b8bbb662 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:43.479402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a9b56-7db9-4c05-b498-7435b8bbb662 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:43.479674 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a9b56-7db9-4c05-b498-7435b8bbb662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:43.504843 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-e26a9b56-7db9-4c05-b498-7435b8bbb662 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:43.506698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e26a9b56-7db9-4c05-b498-7435b8bbb662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:43.517845 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1419/2825] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:43.522208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a9b56-7db9-4c05-b498-7435b8bbb662 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:43.531451 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a9b56-7db9-4c05-b498-7435b8bbb662 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:43.544797 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836a47e-8382-4024-816c-f3ed304d827e None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:43.545586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec63c0a-7649-4253-ab73-793e65b73a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:43.546591 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec63c0a-7649-4253-ab73-793e65b73a77 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:43.546969 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec63c0a-7649-4253-ab73-793e65b73a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:43.547342 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec63c0a-7649-4253-ab73-793e65b73a77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:43.548332 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec63c0a-7649-4253-ab73-793e65b73a77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:32:43.548751 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec63c0a-7649-4253-ab73-793e65b73a77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:43.548869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b586f-bf77-401e-85a6-4c5fd93ce097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:43.549683 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b586f-bf77-401e-85a6-4c5fd93ce097 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:43.550019 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b586f-bf77-401e-85a6-4c5fd93ce097 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:43.550296 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b586f-bf77-401e-85a6-4c5fd93ce097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:43.551063 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec63c0a-7649-4253-ab73-793e65b73a77 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:43.560694 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1b586f-bf77-401e-85a6-4c5fd93ce097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:43.562243 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1b586f-bf77-401e-85a6-4c5fd93ce097 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:43.566385 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1420/2826] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:32:43.571450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec63c0a-7649-4253-ab73-793e65b73a77 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:43.574169 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1407/2827] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:43.583344 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec63c0a-7649-4253-ab73-793e65b73a77 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:43.588800 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e641c-6490-4c7f-85c4-474cc40bc298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:43.589821 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e641c-6490-4c7f-85c4-474cc40bc298 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:43.589959 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e641c-6490-4c7f-85c4-474cc40bc298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:43.590202 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e641c-6490-4c7f-85c4-474cc40bc298 admin admin] PATH_INFO: `/v3/projects` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:43.590868 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e641c-6490-4c7f-85c4-474cc40bc298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:43.592865 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e641c-6490-4c7f-85c4-474cc40bc298 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:43.625856 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1421/2828] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:32:43 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:43.630342 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b586f-bf77-401e-85a6-4c5fd93ce097 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:43.641732 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b586f-bf77-401e-85a6-4c5fd93ce097 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:43.711710 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4a51c-0d4a-43e3-b4e7-4a8a3ec7e6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:43.712666 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4a51c-0d4a-43e3-b4e7-4a8a3ec7e6bc admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:43.713111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4a51c-0d4a-43e3-b4e7-4a8a3ec7e6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:43.713394 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4a51c-0d4a-43e3-b4e7-4a8a3ec7e6bc admin admin] PATH_INFO: `/v3/users` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:43.714115 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e4a51c-0d4a-43e3-b4e7-4a8a3ec7e6bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:43.716466 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e4a51c-0d4a-43e3-b4e7-4a8a3ec7e6bc admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:43.755337 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1408/2829] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:32:43 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:32:43.759954 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e641c-6490-4c7f-85c4-474cc40bc298 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:43.773529 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e641c-6490-4c7f-85c4-474cc40bc298 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:43.778723 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c3321-9a07-4b08-8d22-abac25243b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:43.779394 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c3321-9a07-4b08-8d22-abac25243b72 admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:43.779753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c3321-9a07-4b08-8d22-abac25243b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:43.780478 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c3321-9a07-4b08-8d22-abac25243b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:43.781481 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c3321-9a07-4b08-8d22-abac25243b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:43.783531 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c3321-9a07-4b08-8d22-abac25243b72 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:43.796798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1422/2830] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:43.801237 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4a51c-0d4a-43e3-b4e7-4a8a3ec7e6bc admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:43.812490 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4a51c-0d4a-43e3-b4e7-4a8a3ec7e6bc admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:43.818814 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12babfc3-51a8-4509-a355-526ceedfb050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:43.819495 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12babfc3-51a8-4509-a355-526ceedfb050 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:43.819874 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12babfc3-51a8-4509-a355-526ceedfb050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:43.820188 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12babfc3-51a8-4509-a355-526ceedfb050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:43.820931 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12babfc3-51a8-4509-a355-526ceedfb050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:43.823453 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12babfc3-51a8-4509-a355-526ceedfb050 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:43.839181 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1409/2831] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:43.840793 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c3321-9a07-4b08-8d22-abac25243b72 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:43.851328 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c3321-9a07-4b08-8d22-abac25243b72 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:43.856123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac73ea6-78d6-4695-be04-217c7c0fb4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:43.857991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac73ea6-78d6-4695-be04-217c7c0fb4ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:43.857991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac73ea6-78d6-4695-be04-217c7c0fb4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:43.857991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac73ea6-78d6-4695-be04-217c7c0fb4ca admin admin] PATH_INFO: `/v3/projects/1947572d5cd646d08f4efaa2ef17e271/users/9d3652448ec141918d94617077b9ce81/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:43.877183 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac73ea6-78d6-4695-be04-217c7c0fb4ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=1947572d5cd646d08f4efaa2ef17e271, user_id=9d3652448ec141918d94617077b9ce81, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:43.883531 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac73ea6-78d6-4695-be04-217c7c0fb4ca admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:43.900189 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1423/2832] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:43 2022] PUT /identity/v3/projects/1947572d5cd646d08f4efaa2ef17e271/users/9d3652448ec141918d94617077b9ce81/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:43.905426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c00d64-ebd5-4b62-b908-e7e9cd7e2525 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:43.905870 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c00d64-ebd5-4b62-b908-e7e9cd7e2525 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:43.906122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c00d64-ebd5-4b62-b908-e7e9cd7e2525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:43.928012 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-c8c00d64-ebd5-4b62-b908-e7e9cd7e2525 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:43.929530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c8c00d64-ebd5-4b62-b908-e7e9cd7e2525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:44.007829 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1410/2833] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:44.043551 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac73ea6-78d6-4695-be04-217c7c0fb4ca admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:44.124839 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c00d64-ebd5-4b62-b908-e7e9cd7e2525 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:44.125657 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac73ea6-78d6-4695-be04-217c7c0fb4ca admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:44.126375 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c8c00d64-ebd5-4b62-b908-e7e9cd7e2525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:44.130043 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6c067-275f-4a06-a2ad-c9e3ec533882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:44.130607 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6c067-275f-4a06-a2ad-c9e3ec533882 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:44.131024 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6c067-275f-4a06-a2ad-c9e3ec533882 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:44.131265 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6c067-275f-4a06-a2ad-c9e3ec533882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:44.142477 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6c067-275f-4a06-a2ad-c9e3ec533882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:44.144018 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6c067-275f-4a06-a2ad-c9e3ec533882 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:44.145393 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c00d64-ebd5-4b62-b908-e7e9cd7e2525 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:44.156861 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1424/2834] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:44.163568 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bacc9-8dcd-4ee9-b22c-d54f7a4e57bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:44.164228 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bacc9-8dcd-4ee9-b22c-d54f7a4e57bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:44.164566 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bacc9-8dcd-4ee9-b22c-d54f7a4e57bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:44.164800 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bacc9-8dcd-4ee9-b22c-d54f7a4e57bc admin admin] PATH_INFO: `/v3/projects/7d154ed837c3445691b7baa427dcc89e` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:44.166080 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bacc9-8dcd-4ee9-b22c-d54f7a4e57bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d154ed837c3445691b7baa427dcc89e)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:44.168054 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bacc9-8dcd-4ee9-b22c-d54f7a4e57bc admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:44.186308 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:44.227478 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b78bacc9-8dcd-4ee9-b22c-d54f7a4e57bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d154ed837c3445691b7baa427dcc89e', 'request_id': 'req-b78bacc9-8dcd-4ee9-b22c-d54f7a4e57bc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:44.233639 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b78bacc9-8dcd-4ee9-b22c-d54f7a4e57bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d154ed837c3445691b7baa427dcc89e', 'request_id': 'req-b78bacc9-8dcd-4ee9-b22c-d54f7a4e57bc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:44.242632 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b78bacc9-8dcd-4ee9-b22c-d54f7a4e57bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d154ed837c3445691b7baa427dcc89e', 'request_id': 'req-b78bacc9-8dcd-4ee9-b22c-d54f7a4e57bc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:44.256524 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b78bacc9-8dcd-4ee9-b22c-d54f7a4e57bc admin admin] The token cache is being invalidate because project 7d154ed837c3445691b7baa427dcc89e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:44.256834 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b78bacc9-8dcd-4ee9-b22c-d54f7a4e57bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:44.257495 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b78bacc9-8dcd-4ee9-b22c-d54f7a4e57bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:44.259259 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1411/2835] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:32:44 2022] DELETE /identity/v3/projects/7d154ed837c3445691b7baa427dcc89e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:44.264931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd62b52-f0ae-400b-9363-b3c46c1cdae9 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:44.265216 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd62b52-f0ae-400b-9363-b3c46c1cdae9 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:44.265496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd62b52-f0ae-400b-9363-b3c46c1cdae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:44.290874 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-7dd62b52-f0ae-400b-9363-b3c46c1cdae9 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:44.292336 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-7dd62b52-f0ae-400b-9363-b3c46c1cdae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:44.317419 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:44.381345 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1425/2836] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:44.383445 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ea992-24cb-41c4-b99a-9da01be1c438 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:44.383960 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ea992-24cb-41c4-b99a-9da01be1c438 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:44.384329 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ea992-24cb-41c4-b99a-9da01be1c438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:44.415909 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-884ea992-24cb-41c4-b99a-9da01be1c438 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:44.417324 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-884ea992-24cb-41c4-b99a-9da01be1c438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:44.430745 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1412/2837] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:44.431586 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd62b52-f0ae-400b-9363-b3c46c1cdae9 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:44.442204 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd62b52-f0ae-400b-9363-b3c46c1cdae9 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:44.459208 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb694114-5ce8-4a6f-8b9c-6bd6a307d49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:44.459952 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb694114-5ce8-4a6f-8b9c-6bd6a307d49b admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:44.460231 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb694114-5ce8-4a6f-8b9c-6bd6a307d49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:44.460658 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb694114-5ce8-4a6f-8b9c-6bd6a307d49b admin admin] PATH_INFO: `/v3/domains` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:44.461445 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb694114-5ce8-4a6f-8b9c-6bd6a307d49b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80292) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:32:44.461830 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb694114-5ce8-4a6f-8b9c-6bd6a307d49b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:44.463880 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb694114-5ce8-4a6f-8b9c-6bd6a307d49b admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:44.477496 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1426/2838] 10.222.0.48 () {58 vars in 1153 bytes} [Mon Aug 1 15:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:32:44.482669 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ea992-24cb-41c4-b99a-9da01be1c438 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:44.493272 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ea992-24cb-41c4-b99a-9da01be1c438 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:44.498515 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7352e7a-d5b6-4b3c-b56b-31f9d7dad5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:44.499151 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7352e7a-d5b6-4b3c-b56b-31f9d7dad5e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:44.499424 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7352e7a-d5b6-4b3c-b56b-31f9d7dad5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:44.499919 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7352e7a-d5b6-4b3c-b56b-31f9d7dad5e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:44.500383 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7352e7a-d5b6-4b3c-b56b-31f9d7dad5e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:44.502370 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7352e7a-d5b6-4b3c-b56b-31f9d7dad5e6 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:44.546081 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1413/2839] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:32:44 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:44.547357 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb694114-5ce8-4a6f-8b9c-6bd6a307d49b admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:44.558931 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb694114-5ce8-4a6f-8b9c-6bd6a307d49b admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:44.625232 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366aa266-7354-4513-8d0c-47f2bc9ddc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:44.626100 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366aa266-7354-4513-8d0c-47f2bc9ddc23 admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:44.626656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366aa266-7354-4513-8d0c-47f2bc9ddc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:44.626656 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366aa266-7354-4513-8d0c-47f2bc9ddc23 admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:44.627620 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366aa266-7354-4513-8d0c-47f2bc9ddc23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:44.629928 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366aa266-7354-4513-8d0c-47f2bc9ddc23 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:44.655491 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1427/2840] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:32:44 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:32:44.659729 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7352e7a-d5b6-4b3c-b56b-31f9d7dad5e6 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:44.671495 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7352e7a-d5b6-4b3c-b56b-31f9d7dad5e6 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:44.678248 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b167fc-4e42-4b28-b5fb-0852209ca164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:44.678248 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b167fc-4e42-4b28-b5fb-0852209ca164 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:44.678685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b167fc-4e42-4b28-b5fb-0852209ca164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:44.678685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b167fc-4e42-4b28-b5fb-0852209ca164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:44.679833 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b167fc-4e42-4b28-b5fb-0852209ca164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:44.681020 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b167fc-4e42-4b28-b5fb-0852209ca164 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:44.696178 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1414/2841] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:44.702052 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366aa266-7354-4513-8d0c-47f2bc9ddc23 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:44.730876 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366aa266-7354-4513-8d0c-47f2bc9ddc23 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:44.747952 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fac020-df5d-4e12-b069-48c0f56f207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:44.748784 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fac020-df5d-4e12-b069-48c0f56f207f admin admin] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:44.749068 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fac020-df5d-4e12-b069-48c0f56f207f admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:44.749403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fac020-df5d-4e12-b069-48c0f56f207f admin admin] PATH_INFO: `/v3/roles` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:44.750223 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fac020-df5d-4e12-b069-48c0f56f207f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:44.753717 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fac020-df5d-4e12-b069-48c0f56f207f admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:44.772261 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1428/2842] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:44.777024 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b167fc-4e42-4b28-b5fb-0852209ca164 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:44.787836 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b167fc-4e42-4b28-b5fb-0852209ca164 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:44.793141 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e087d9-70bf-451c-ba73-11b08a9efb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:44.794361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e087d9-70bf-451c-ba73-11b08a9efb1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:44.794828 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e087d9-70bf-451c-ba73-11b08a9efb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:44.795106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e087d9-70bf-451c-ba73-11b08a9efb1c admin admin] PATH_INFO: `/v3/projects/ff70f42c96804182af2e61e501ee8a21/users/5f2b4bba3ed5400d8ab618a6ccbc04d4/roles/19db9fb1cb1541ee94673df232bd6952` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:44.816831 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e087d9-70bf-451c-ba73-11b08a9efb1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff70f42c96804182af2e61e501ee8a21, user_id=5f2b4bba3ed5400d8ab618a6ccbc04d4, role_id=19db9fb1cb1541ee94673df232bd6952)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:44.820302 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e087d9-70bf-451c-ba73-11b08a9efb1c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:44.839116 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1415/2843] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:44 2022] PUT /identity/v3/projects/ff70f42c96804182af2e61e501ee8a21/users/5f2b4bba3ed5400d8ab618a6ccbc04d4/roles/19db9fb1cb1541ee94673df232bd6952 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:44.844566 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462054fb-bbb1-4bd5-a292-ec8c3430b233 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:44.844858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462054fb-bbb1-4bd5-a292-ec8c3430b233 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:44.845156 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462054fb-bbb1-4bd5-a292-ec8c3430b233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:44.867912 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-462054fb-bbb1-4bd5-a292-ec8c3430b233 None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:44.869587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-462054fb-bbb1-4bd5-a292-ec8c3430b233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:44.948469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1429/2844] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:44.962089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e087d9-70bf-451c-ba73-11b08a9efb1c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:44.964038 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-d2e087d9-70bf-451c-ba73-11b08a9efb1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:45.091318 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e087d9-70bf-451c-ba73-11b08a9efb1c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:45.101224 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a6d90-5472-4634-a154-c44fdde891a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:45.102196 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a6d90-5472-4634-a154-c44fdde891a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:45.102801 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a6d90-5472-4634-a154-c44fdde891a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:45.103303 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a6d90-5472-4634-a154-c44fdde891a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:45.119051 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a6d90-5472-4634-a154-c44fdde891a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:45.120862 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a6d90-5472-4634-a154-c44fdde891a8 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:45.136529 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1416/2845] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:45.969224 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b7c73-4c1b-476d-b807-9930c38827b0 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:45.969866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b7c73-4c1b-476d-b807-9930c38827b0 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:45.970053 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b7c73-4c1b-476d-b807-9930c38827b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:46.005358 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-7d6b7c73-4c1b-476d-b807-9930c38827b0 None None] MFA Rules not processed for user `9d3652448ec141918d94617077b9ce81`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:46.006999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-7d6b7c73-4c1b-476d-b807-9930c38827b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:46.043550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:46.136760 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1430/2846] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:46.149967 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a6d90-5472-4634-a154-c44fdde891a8 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:46.152466 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-652a6d90-5472-4634-a154-c44fdde891a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:46.183737 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b7c73-4c1b-476d-b807-9930c38827b0 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:46.202601 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b7c73-4c1b-476d-b807-9930c38827b0 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:46.208012 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825beba-58a6-474d-8c11-1dbf51408ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:46.209851 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825beba-58a6-474d-8c11-1dbf51408ddc service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:46.211074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825beba-58a6-474d-8c11-1dbf51408ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:46.211930 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825beba-58a6-474d-8c11-1dbf51408ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:46.213058 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:46.217274 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f825beba-58a6-474d-8c11-1dbf51408ddc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:46.229214 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a6d90-5472-4634-a154-c44fdde891a8 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:46.235117 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375ada7-4bc2-4932-b5b8-8d2250687972 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '63824ba9369c4899a4b946588cbdae24', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:46.236063 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375ada7-4bc2-4932-b5b8-8d2250687972 service cinder] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:46.236501 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375ada7-4bc2-4932-b5b8-8d2250687972 service cinder] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:46.236930 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375ada7-4bc2-4932-b5b8-8d2250687972 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:46.238305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825beba-58a6-474d-8c11-1dbf51408ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:46.241640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825beba-58a6-474d-8c11-1dbf51408ddc service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:46.248257 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375ada7-4bc2-4932-b5b8-8d2250687972 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:46.250384 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375ada7-4bc2-4932-b5b8-8d2250687972 service cinder] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:46.261503 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1417/2847] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:46.264069 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1431/2848] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:47.853642 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dd1ea-dec8-4f98-9385-60a709e113b5 None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:47.854223 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dd1ea-dec8-4f98-9385-60a709e113b5 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:47.854223 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dd1ea-dec8-4f98-9385-60a709e113b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:47.886709 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-4d0dd1ea-dec8-4f98-9385-60a709e113b5 None None] MFA Rules not processed for user `9d3652448ec141918d94617077b9ce81`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:47.888073 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4d0dd1ea-dec8-4f98-9385-60a709e113b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:47.912930 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1418/2849] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:52.358546 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9e227-9afb-4f59-bee9-165a92801a99 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:52.359097 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9e227-9afb-4f59-bee9-165a92801a99 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:52.359548 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9e227-9afb-4f59-bee9-165a92801a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:52.453999 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-f8c9e227-9afb-4f59-bee9-165a92801a99 None None] MFA Rules not processed for user `5f2b4bba3ed5400d8ab618a6ccbc04d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:52.457403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-f8c9e227-9afb-4f59-bee9-165a92801a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:52.569330 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1432/2850] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:32:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:52.570748 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0dd1ea-dec8-4f98-9385-60a709e113b5 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:52.584449 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0dd1ea-dec8-4f98-9385-60a709e113b5 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:52.595710 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d71ef-3b83-4f23-b9ec-4ff536f40d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:52.596362 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d71ef-3b83-4f23-b9ec-4ff536f40d0c admin admin] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:52.596640 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d71ef-3b83-4f23-b9ec-4ff536f40d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:52.596914 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d71ef-3b83-4f23-b9ec-4ff536f40d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:52.597588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d71ef-3b83-4f23-b9ec-4ff536f40d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:52.599835 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d71ef-3b83-4f23-b9ec-4ff536f40d0c admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:52.633820 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1419/2851] 10.222.0.48 () {60 vars in 1153 bytes} [Mon Aug 1 15:32:52 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:52.636247 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9e227-9afb-4f59-bee9-165a92801a99 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:52.649216 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9e227-9afb-4f59-bee9-165a92801a99 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:52.722320 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf146ba4-250c-4307-a3de-a65b7c910afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:52.723270 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf146ba4-250c-4307-a3de-a65b7c910afd admin admin] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:52.723637 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf146ba4-250c-4307-a3de-a65b7c910afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:52.723979 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf146ba4-250c-4307-a3de-a65b7c910afd admin admin] PATH_INFO: `/v3/users` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:52.724779 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf146ba4-250c-4307-a3de-a65b7c910afd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:52.727195 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf146ba4-250c-4307-a3de-a65b7c910afd admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:52.761818 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1433/2852] 10.222.0.48 () {60 vars in 1144 bytes} [Mon Aug 1 15:32:52 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:52.766346 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d71ef-3b83-4f23-b9ec-4ff536f40d0c admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:52.778047 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d71ef-3b83-4f23-b9ec-4ff536f40d0c admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:52.783485 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62d19b-e86e-4d35-b262-39af69e3c1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:52.784141 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62d19b-e86e-4d35-b262-39af69e3c1c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:52.784415 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62d19b-e86e-4d35-b262-39af69e3c1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:52.784685 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62d19b-e86e-4d35-b262-39af69e3c1c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:52.785396 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62d19b-e86e-4d35-b262-39af69e3c1c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:52.787133 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62d19b-e86e-4d35-b262-39af69e3c1c7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:52.802859 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1420/2853] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:52.804055 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf146ba4-250c-4307-a3de-a65b7c910afd admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:52.815559 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf146ba4-250c-4307-a3de-a65b7c910afd admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:52.822664 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351657d-a843-4c64-a8f3-452b4e156264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:52.823618 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351657d-a843-4c64-a8f3-452b4e156264 admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:52.823900 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351657d-a843-4c64-a8f3-452b4e156264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:52.824284 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351657d-a843-4c64-a8f3-452b4e156264 admin admin] PATH_INFO: `/v3/projects/a5317b74e40d4a8fa5b0cc2beb9e005c/users/e97c98ea43e9441d9e0ba3051410a577/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:52.843323 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351657d-a843-4c64-a8f3-452b4e156264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5317b74e40d4a8fa5b0cc2beb9e005c, user_id=e97c98ea43e9441d9e0ba3051410a577, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:52.846354 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351657d-a843-4c64-a8f3-452b4e156264 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:52.859661 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:52.871034 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1434/2854] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:52 2022] PUT /identity/v3/projects/a5317b74e40d4a8fa5b0cc2beb9e005c/users/e97c98ea43e9441d9e0ba3051410a577/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:52.880556 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62d19b-e86e-4d35-b262-39af69e3c1c7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:52.898573 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62d19b-e86e-4d35-b262-39af69e3c1c7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:52.974562 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a9965-3454-4b2c-bbba-f4365f23a450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:52.975209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a9965-3454-4b2c-bbba-f4365f23a450 admin admin] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:52.975491 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a9965-3454-4b2c-bbba-f4365f23a450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:52.975764 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a9965-3454-4b2c-bbba-f4365f23a450 admin admin] PATH_INFO: `/v3/roles` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:52.976505 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a9965-3454-4b2c-bbba-f4365f23a450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:52.978284 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a9965-3454-4b2c-bbba-f4365f23a450 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:52.989897 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1421/2855] 10.222.0.48 () {58 vars in 1122 bytes} [Mon Aug 1 15:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:52.996660 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351657d-a843-4c64-a8f3-452b4e156264 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:53.009582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351657d-a843-4c64-a8f3-452b4e156264 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:53.016062 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd86bb-4a51-4ff2-b243-41446f0015bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:53.017021 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd86bb-4a51-4ff2-b243-41446f0015bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:53.017456 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd86bb-4a51-4ff2-b243-41446f0015bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:53.017940 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd86bb-4a51-4ff2-b243-41446f0015bf admin admin] PATH_INFO: `/v3/projects/a5317b74e40d4a8fa5b0cc2beb9e005c/users/e97c98ea43e9441d9e0ba3051410a577/roles/4db5fcc48be54ef597f2e2fffeae9018` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:53.020199 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd86bb-4a51-4ff2-b243-41446f0015bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5317b74e40d4a8fa5b0cc2beb9e005c, user_id=e97c98ea43e9441d9e0ba3051410a577, role_id=4db5fcc48be54ef597f2e2fffeae9018)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:53.023165 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd86bb-4a51-4ff2-b243-41446f0015bf admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:53.035180 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1435/2856] 10.222.0.48 () {60 vars in 1483 bytes} [Mon Aug 1 15:32:52 2022] PUT /identity/v3/projects/a5317b74e40d4a8fa5b0cc2beb9e005c/users/e97c98ea43e9441d9e0ba3051410a577/roles/4db5fcc48be54ef597f2e2fffeae9018 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:59.213178 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4eb39-ba1c-43e9-ac65-e6afff974af4 None None] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:59.216783 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4eb39-ba1c-43e9-ac65-e6afff974af4 None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:59.218926 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4eb39-ba1c-43e9-ac65-e6afff974af4 None None] PATH_INFO: `/` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:59.220640 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1422/2857] 10.222.0.48 () {54 vars in 921 bytes} [Mon Aug 1 15:32:59 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 15:32:59.258461 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50262eef-0078-4eab-a1a3-44aec88d7b79 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:59.259738 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50262eef-0078-4eab-a1a3-44aec88d7b79 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:59.260287 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50262eef-0078-4eab-a1a3-44aec88d7b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:59.377446 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4eb39-ba1c-43e9-ac65-e6afff974af4 None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:59.379622 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-84b4eb39-ba1c-43e9-ac65-e6afff974af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:59.431631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-50262eef-0078-4eab-a1a3-44aec88d7b79 None None] MFA Rules not processed for user `098d8d7e35804ccc931ddfba07f688a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:59.432625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-50262eef-0078-4eab-a1a3-44aec88d7b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:59.546039 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1436/2858] 10.222.0.48 () {58 vars in 1018 bytes} [Mon Aug 1 15:32:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:59.597678 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50262eef-0078-4eab-a1a3-44aec88d7b79 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:59.599222 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-50262eef-0078-4eab-a1a3-44aec88d7b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:59.642819 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50262eef-0078-4eab-a1a3-44aec88d7b79 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:59.656625 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e43475-2872-4f41-b939-fd4b92e4285a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '098d8d7e35804ccc931ddfba07f688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:59.657554 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e43475-2872-4f41-b939-fd4b92e4285a service placement] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:59.658030 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e43475-2872-4f41-b939-fd4b92e4285a service placement] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:59.659957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e43475-2872-4f41-b939-fd4b92e4285a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:59.676406 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e43475-2872-4f41-b939-fd4b92e4285a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:59.678838 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e43475-2872-4f41-b939-fd4b92e4285a service placement] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:59.739896 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4eb39-ba1c-43e9-ac65-e6afff974af4 None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:59.746720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d94e0-ddd4-458a-952d-927c89b20480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:59.747957 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d94e0-ddd4-458a-952d-927c89b20480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:59.748424 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d94e0-ddd4-458a-952d-927c89b20480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:59.748881 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d94e0-ddd4-458a-952d-927c89b20480 admin admin] PATH_INFO: `/v3/users/30b9685e174a4523a6c36d4d0bebd5fc` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:59.751326 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d94e0-ddd4-458a-952d-927c89b20480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30b9685e174a4523a6c36d4d0bebd5fc)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:59.753982 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d94e0-ddd4-458a-952d-927c89b20480 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:59.756871 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1437/2859] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:32:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:32:59.867117 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:59.880923 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ed5d94e0-ddd4-458a-952d-927c89b20480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30b9685e174a4523a6c36d4d0bebd5fc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:59.882924 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ed5d94e0-ddd4-458a-952d-927c89b20480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30b9685e174a4523a6c36d4d0bebd5fc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:59.921279 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ed5d94e0-ddd4-458a-952d-927c89b20480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30b9685e174a4523a6c36d4d0bebd5fc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:59.946988 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ed5d94e0-ddd4-458a-952d-927c89b20480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30b9685e174a4523a6c36d4d0bebd5fc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:59.982620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-ed5d94e0-ddd4-458a-952d-927c89b20480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30b9685e174a4523a6c36d4d0bebd5fc'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:59.999399 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1423/2860] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:32:59 2022] DELETE /identity/v3/users/30b9685e174a4523a6c36d4d0bebd5fc => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:33:00.123679 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2a4e0-e47f-4c84-84e5-86cfe570e875 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:00.123965 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2a4e0-e47f-4c84-84e5-86cfe570e875 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:00.124244 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2a4e0-e47f-4c84-84e5-86cfe570e875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:00.163015 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-86a2a4e0-e47f-4c84-84e5-86cfe570e875 None None] MFA Rules not processed for user `e97c98ea43e9441d9e0ba3051410a577`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:00.165073 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-86a2a4e0-e47f-4c84-84e5-86cfe570e875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:00.248167 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d94e0-ddd4-458a-952d-927c89b20480 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:00.251614 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-ed5d94e0-ddd4-458a-952d-927c89b20480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:00.291602 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1438/2861] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:33:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:00.305708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2a4e0-e47f-4c84-84e5-86cfe570e875 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:00.316011 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-86a2a4e0-e47f-4c84-84e5-86cfe570e875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:00.362782 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d94e0-ddd4-458a-952d-927c89b20480 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:00.368547 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b374d8-88c6-41e4-ae18-a8714b25d29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:00.370334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b374d8-88c6-41e4-ae18-a8714b25d29b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:00.371095 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b374d8-88c6-41e4-ae18-a8714b25d29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:00.371766 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b374d8-88c6-41e4-ae18-a8714b25d29b admin admin] PATH_INFO: `/v3/projects/4ad998384c894fe9aed6d57f7aa1110b` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:00.373611 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b374d8-88c6-41e4-ae18-a8714b25d29b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ad998384c894fe9aed6d57f7aa1110b)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:00.376053 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b374d8-88c6-41e4-ae18-a8714b25d29b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:00.412462 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:00.449218 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-83b374d8-88c6-41e4-ae18-a8714b25d29b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ad998384c894fe9aed6d57f7aa1110b', 'request_id': 'req-83b374d8-88c6-41e4-ae18-a8714b25d29b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:00.451092 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2a4e0-e47f-4c84-84e5-86cfe570e875 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:00.460420 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-83b374d8-88c6-41e4-ae18-a8714b25d29b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ad998384c894fe9aed6d57f7aa1110b', 'request_id': 'req-83b374d8-88c6-41e4-ae18-a8714b25d29b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:00.487363 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-83b374d8-88c6-41e4-ae18-a8714b25d29b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ad998384c894fe9aed6d57f7aa1110b', 'request_id': 'req-83b374d8-88c6-41e4-ae18-a8714b25d29b'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:00.516206 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-83b374d8-88c6-41e4-ae18-a8714b25d29b admin admin] The token cache is being invalidate because project 4ad998384c894fe9aed6d57f7aa1110b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:33:00.517480 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-83b374d8-88c6-41e4-ae18-a8714b25d29b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:00.518186 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-83b374d8-88c6-41e4-ae18-a8714b25d29b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:00.522212 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1424/2862] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:33:00 2022] DELETE /identity/v3/projects/4ad998384c894fe9aed6d57f7aa1110b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:00.523150 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884177cf-c172-4805-ae6e-0902e6fdf9cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94f696dfe4b34c4aa19104b5f7ee6e70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:00.523909 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884177cf-c172-4805-ae6e-0902e6fdf9cb service glance] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:00.525426 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b374d8-88c6-41e4-ae18-a8714b25d29b admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:00.526337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884177cf-c172-4805-ae6e-0902e6fdf9cb service glance] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:00.526890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884177cf-c172-4805-ae6e-0902e6fdf9cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:00.527012 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-83b374d8-88c6-41e4-ae18-a8714b25d29b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:00.529151 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-884177cf-c172-4805-ae6e-0902e6fdf9cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:00.600313 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884177cf-c172-4805-ae6e-0902e6fdf9cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:00.601957 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884177cf-c172-4805-ae6e-0902e6fdf9cb service glance] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:00.610248 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:00.613742 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1439/2863] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:33:00.614488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884177cf-c172-4805-ae6e-0902e6fdf9cb service glance] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:00.615889 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-884177cf-c172-4805-ae6e-0902e6fdf9cb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:00.630676 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b374d8-88c6-41e4-ae18-a8714b25d29b admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:00.632719 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884177cf-c172-4805-ae6e-0902e6fdf9cb service glance] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:00.638600 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7d52f1-cb7f-4fd5-ba69-4eac9aa1f895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:00.639738 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d52f1-cb7f-4fd5-ba69-4eac9aa1f895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:00.640403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d52f1-cb7f-4fd5-ba69-4eac9aa1f895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:00.640403 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d52f1-cb7f-4fd5-ba69-4eac9aa1f895 admin admin] PATH_INFO: `/v3/users/d79de80f35634aa78e04a5765ae8f99f` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:00.641647 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d52f1-cb7f-4fd5-ba69-4eac9aa1f895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d79de80f35634aa78e04a5765ae8f99f)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:00.642112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6331ee-2019-475a-bb48-e3d4b3e4c42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:00.643000 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6331ee-2019-475a-bb48-e3d4b3e4c42b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:00.643470 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6331ee-2019-475a-bb48-e3d4b3e4c42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:00.643965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6331ee-2019-475a-bb48-e3d4b3e4c42b admin admin] PATH_INFO: `/v3/users/782acde280984ec6a01d788c95754f35` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:00.644118 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7d52f1-cb7f-4fd5-ba69-4eac9aa1f895 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:00.645825 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6331ee-2019-475a-bb48-e3d4b3e4c42b admin admin] RBAC: Authorizing `identity:delete_user(user_id=782acde280984ec6a01d788c95754f35)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:00.648106 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6331ee-2019-475a-bb48-e3d4b3e4c42b admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:00.725982 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0d7d52f1-cb7f-4fd5-ba69-4eac9aa1f895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd79de80f35634aa78e04a5765ae8f99f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:00.730461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9b6331ee-2019-475a-bb48-e3d4b3e4c42b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '782acde280984ec6a01d788c95754f35'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:00.731725 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9b6331ee-2019-475a-bb48-e3d4b3e4c42b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '782acde280984ec6a01d788c95754f35'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:00.739063 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0d7d52f1-cb7f-4fd5-ba69-4eac9aa1f895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd79de80f35634aa78e04a5765ae8f99f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:00.739936 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0d7d52f1-cb7f-4fd5-ba69-4eac9aa1f895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd79de80f35634aa78e04a5765ae8f99f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:00.749652 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0d7d52f1-cb7f-4fd5-ba69-4eac9aa1f895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd79de80f35634aa78e04a5765ae8f99f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:00.753241 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9b6331ee-2019-475a-bb48-e3d4b3e4c42b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '782acde280984ec6a01d788c95754f35'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:00.766773 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9b6331ee-2019-475a-bb48-e3d4b3e4c42b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '782acde280984ec6a01d788c95754f35'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:00.767483 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-0d7d52f1-cb7f-4fd5-ba69-4eac9aa1f895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd79de80f35634aa78e04a5765ae8f99f'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:00.776518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-9b6331ee-2019-475a-bb48-e3d4b3e4c42b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '782acde280984ec6a01d788c95754f35'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:00.777906 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1440/2864] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:33:00 2022] DELETE /identity/v3/users/d79de80f35634aa78e04a5765ae8f99f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:00.788570 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604322c9-f25b-4f82-bcee-892c5790e4d3 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:00.789307 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604322c9-f25b-4f82-bcee-892c5790e4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:00.789646 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604322c9-f25b-4f82-bcee-892c5790e4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:00.805356 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1425/2865] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:33:00 2022] DELETE /identity/v3/users/782acde280984ec6a01d788c95754f35 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:00.811966 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbf501-7456-4cf9-93a2-5317cd4df5ac None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:00.812346 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbf501-7456-4cf9-93a2-5317cd4df5ac None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:00.812662 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbf501-7456-4cf9-93a2-5317cd4df5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:00.821446 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-604322c9-f25b-4f82-bcee-892c5790e4d3 None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:00.828279 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-604322c9-f25b-4f82-bcee-892c5790e4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:00.853353 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-e5bbf501-7456-4cf9-93a2-5317cd4df5ac None None] MFA Rules not processed for user `1b4b6acdd88e421a86924fd2c155794f`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:00.855488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:00.857079 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e5bbf501-7456-4cf9-93a2-5317cd4df5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:00.938934 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1441/2866] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 15:33:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:33:00.960209 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1426/2867] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:33:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:00.973858 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604322c9-f25b-4f82-bcee-892c5790e4d3 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:00.975576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-604322c9-f25b-4f82-bcee-892c5790e4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:01.019939 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbf501-7456-4cf9-93a2-5317cd4df5ac None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:01.021620 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-e5bbf501-7456-4cf9-93a2-5317cd4df5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:01.078277 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604322c9-f25b-4f82-bcee-892c5790e4d3 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:01.087240 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4b56c-b9b1-4eea-a810-b24446059d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:01.088682 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4b56c-b9b1-4eea-a810-b24446059d30 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:01.089163 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4b56c-b9b1-4eea-a810-b24446059d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:01.089452 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4b56c-b9b1-4eea-a810-b24446059d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:01.100854 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf4b56c-b9b1-4eea-a810-b24446059d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:01.102450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf4b56c-b9b1-4eea-a810-b24446059d30 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:01.114048 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1442/2868] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:01.141744 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbf501-7456-4cf9-93a2-5317cd4df5ac None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:01.147080 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4b56c-b9b1-4eea-a810-b24446059d30 service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:01.149759 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-4cf4b56c-b9b1-4eea-a810-b24446059d30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:01.167074 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4b56c-b9b1-4eea-a810-b24446059d30 service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:01.168345 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c265f09-3cad-444b-b157-f07700759fc6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:01.168857 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c265f09-3cad-444b-b157-f07700759fc6 service swift] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:01.169221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c265f09-3cad-444b-b157-f07700759fc6 service swift] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:01.169709 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c265f09-3cad-444b-b157-f07700759fc6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:01.181144 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5d1a5-badb-431f-8277-ce0a3ec6d542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:01.181771 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c265f09-3cad-444b-b157-f07700759fc6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:01.182615 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5d1a5-badb-431f-8277-ce0a3ec6d542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:01.182830 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5d1a5-badb-431f-8277-ce0a3ec6d542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:01.183112 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5d1a5-badb-431f-8277-ce0a3ec6d542 admin admin] PATH_INFO: `/v3/projects/a8d4f982f0664ab18c1a6352d0c07c66` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:01.183733 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c265f09-3cad-444b-b157-f07700759fc6 service swift] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:01.184650 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5d1a5-badb-431f-8277-ce0a3ec6d542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8d4f982f0664ab18c1a6352d0c07c66)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:01.187879 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5d1a5-badb-431f-8277-ce0a3ec6d542 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:01.189916 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:01.195671 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1427/2869] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:33:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:33:01.269844 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-58a5d1a5-badb-431f-8277-ce0a3ec6d542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d4f982f0664ab18c1a6352d0c07c66', 'request_id': 'req-58a5d1a5-badb-431f-8277-ce0a3ec6d542'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:01.290753 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-58a5d1a5-badb-431f-8277-ce0a3ec6d542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d4f982f0664ab18c1a6352d0c07c66', 'request_id': 'req-58a5d1a5-badb-431f-8277-ce0a3ec6d542'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:01.300387 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-58a5d1a5-badb-431f-8277-ce0a3ec6d542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d4f982f0664ab18c1a6352d0c07c66', 'request_id': 'req-58a5d1a5-badb-431f-8277-ce0a3ec6d542'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:01.315787 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-58a5d1a5-badb-431f-8277-ce0a3ec6d542 admin admin] The token cache is being invalidate because project a8d4f982f0664ab18c1a6352d0c07c66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80292) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:33:01.316350 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-58a5d1a5-badb-431f-8277-ce0a3ec6d542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:01.317389 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-58a5d1a5-badb-431f-8277-ce0a3ec6d542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:01.319289 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1443/2870] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:33:01 2022] DELETE /identity/v3/projects/a8d4f982f0664ab18c1a6352d0c07c66 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:01.727917 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c265f09-3cad-444b-b157-f07700759fc6 service swift] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:01.729465 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4c265f09-3cad-444b-b157-f07700759fc6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:01.761377 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c265f09-3cad-444b-b157-f07700759fc6 service swift] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:01.870324 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db8c205-8f2c-4f48-b975-5232adb7d278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:01.871004 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8c205-8f2c-4f48-b975-5232adb7d278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:01.875129 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8c205-8f2c-4f48-b975-5232adb7d278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:01.875461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db8c205-8f2c-4f48-b975-5232adb7d278 admin admin] PATH_INFO: `/v3/projects/5cdf1fff648c440eb3b69dc25b5d8dc6` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:01.877438 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8c205-8f2c-4f48-b975-5232adb7d278 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cdf1fff648c440eb3b69dc25b5d8dc6)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:01.879526 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db8c205-8f2c-4f48-b975-5232adb7d278 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:01.977493 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8db8c205-8f2c-4f48-b975-5232adb7d278 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cdf1fff648c440eb3b69dc25b5d8dc6', 'request_id': 'req-8db8c205-8f2c-4f48-b975-5232adb7d278'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:01.993798 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8db8c205-8f2c-4f48-b975-5232adb7d278 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cdf1fff648c440eb3b69dc25b5d8dc6', 'request_id': 'req-8db8c205-8f2c-4f48-b975-5232adb7d278'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:02.005965 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8db8c205-8f2c-4f48-b975-5232adb7d278 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cdf1fff648c440eb3b69dc25b5d8dc6', 'request_id': 'req-8db8c205-8f2c-4f48-b975-5232adb7d278'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:02.029878 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8db8c205-8f2c-4f48-b975-5232adb7d278 admin admin] The token cache is being invalidate because project 5cdf1fff648c440eb3b69dc25b5d8dc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:33:02.030253 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8db8c205-8f2c-4f48-b975-5232adb7d278 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:02.031121 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-8db8c205-8f2c-4f48-b975-5232adb7d278 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:02.035207 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1428/2871] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:33:01 2022] DELETE /identity/v3/projects/5cdf1fff648c440eb3b69dc25b5d8dc6 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:05.644223 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602e4c4-f12c-4748-8181-7b89181da3db None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:05.644766 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602e4c4-f12c-4748-8181-7b89181da3db None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:05.644891 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602e4c4-f12c-4748-8181-7b89181da3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:05.672690 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-e602e4c4-f12c-4748-8181-7b89181da3db None None] MFA Rules not processed for user `9d3652448ec141918d94617077b9ce81`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:05.674329 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-e602e4c4-f12c-4748-8181-7b89181da3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:05.789323 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:05.812903 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1444/2872] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:33:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:11.003654 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda34e6a-9438-4627-8817-2b46bc63d54c None None] REQUEST_METHOD: `POST` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:11.004177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda34e6a-9438-4627-8817-2b46bc63d54c None None] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:11.004177 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda34e6a-9438-4627-8817-2b46bc63d54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:11.024882 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.auth.core [None req-bda34e6a-9438-4627-8817-2b46bc63d54c None None] MFA Rules not processed for user `5f2b4bba3ed5400d8ab618a6ccbc04d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80291) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:11.026530 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bda34e6a-9438-4627-8817-2b46bc63d54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:11.095334 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1429/2873] 10.222.0.48 () {58 vars in 958 bytes} [Mon Aug 1 15:33:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:33:12.929096 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967c615-1314-4399-9046-c7f066b8f01f None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:12.929574 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967c615-1314-4399-9046-c7f066b8f01f None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:12.929974 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967c615-1314-4399-9046-c7f066b8f01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:12.949330 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-b967c615-1314-4399-9046-c7f066b8f01f None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:12.950758 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-b967c615-1314-4399-9046-c7f066b8f01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:13.013556 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1445/2874] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 15:33:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:13.026331 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda34e6a-9438-4627-8817-2b46bc63d54c None None] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:13.028030 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-bda34e6a-9438-4627-8817-2b46bc63d54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:13.068393 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:13.116509 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda34e6a-9438-4627-8817-2b46bc63d54c None None] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:13.122412 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5bdba-cc85-4aac-bc81-a662c3ce631d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:13.123041 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5bdba-cc85-4aac-bc81-a662c3ce631d service swift] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:13.123319 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5bdba-cc85-4aac-bc81-a662c3ce631d service swift] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:13.123588 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5bdba-cc85-4aac-bc81-a662c3ce631d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:13.132581 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5bdba-cc85-4aac-bc81-a662c3ce631d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:13.135135 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5bdba-cc85-4aac-bc81-a662c3ce631d service swift] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:13.144344 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1430/2875] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:33:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:33:23.006664 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=29 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:23.140933 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=16 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:34.926000 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33e385-fdc9-496e-8c09-0faf21df5384 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:34.926570 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33e385-fdc9-496e-8c09-0faf21df5384 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:34.926570 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33e385-fdc9-496e-8c09-0faf21df5384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:34.949470 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-1b33e385-fdc9-496e-8c09-0faf21df5384 None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:34.951771 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1b33e385-fdc9-496e-8c09-0faf21df5384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:34.963555 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1446/2876] 10.222.0.48 () {58 vars in 1009 bytes} [Mon Aug 1 15:33:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:33:34.979822 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5bdba-cc85-4aac-bc81-a662c3ce631d service swift] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:34.992292 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5bdba-cc85-4aac-bc81-a662c3ce631d service swift] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:34.998604 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62494d17-37ae-4be4-8246-3bf3266ce76b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:34.999730 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62494d17-37ae-4be4-8246-3bf3266ce76b service swift] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:35.000172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62494d17-37ae-4be4-8246-3bf3266ce76b service swift] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:35.000605 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62494d17-37ae-4be4-8246-3bf3266ce76b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:35.011393 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62494d17-37ae-4be4-8246-3bf3266ce76b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:35.013089 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62494d17-37ae-4be4-8246-3bf3266ce76b service swift] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:35.022969 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1431/2877] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:33:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:33:44.026384 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33e385-fdc9-496e-8c09-0faf21df5384 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:44.034652 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1b33e385-fdc9-496e-8c09-0faf21df5384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:44.115227 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33e385-fdc9-496e-8c09-0faf21df5384 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:44.119194 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456d27c-3964-446b-8c73-1b0c772dd17c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:44.119979 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456d27c-3964-446b-8c73-1b0c772dd17c service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:44.120315 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456d27c-3964-446b-8c73-1b0c772dd17c service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:44.120640 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456d27c-3964-446b-8c73-1b0c772dd17c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.122665 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1456d27c-3964-446b-8c73-1b0c772dd17c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:44.222757 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456d27c-3964-446b-8c73-1b0c772dd17c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:44.224545 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456d27c-3964-446b-8c73-1b0c772dd17c service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:44.238533 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1447/2878] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:44.667295 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62494d17-37ae-4be4-8246-3bf3266ce76b service swift] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:44.684287 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62494d17-37ae-4be4-8246-3bf3266ce76b service swift] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:44.690627 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a0693-3489-4e71-87e4-4ddee991cd57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:44.691433 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a0693-3489-4e71-87e4-4ddee991cd57 service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:44.691832 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a0693-3489-4e71-87e4-4ddee991cd57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:44.692225 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a0693-3489-4e71-87e4-4ddee991cd57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:44.694586 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-c56a0693-3489-4e71-87e4-4ddee991cd57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:44.822354 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a0693-3489-4e71-87e4-4ddee991cd57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:44.823772 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a0693-3489-4e71-87e4-4ddee991cd57 service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:44.853840 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1432/2879] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:48.813709 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456d27c-3964-446b-8c73-1b0c772dd17c service neutron] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:48.826646 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456d27c-3964-446b-8c73-1b0c772dd17c service neutron] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:48.834959 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7613c8-211d-45d7-80a9-ad5b042ad4d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:48.836072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7613c8-211d-45d7-80a9-ad5b042ad4d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:48.836350 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7613c8-211d-45d7-80a9-ad5b042ad4d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:48.836827 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7613c8-211d-45d7-80a9-ad5b042ad4d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:48.840495 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-1d7613c8-211d-45d7-80a9-ad5b042ad4d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:48.994450 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7613c8-211d-45d7-80a9-ad5b042ad4d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:48.996382 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7613c8-211d-45d7-80a9-ad5b042ad4d3 service neutron] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:49.011396 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1448/2880] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:49.937431 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a0693-3489-4e71-87e4-4ddee991cd57 service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:49.947987 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a0693-3489-4e71-87e4-4ddee991cd57 service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:49.952250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a761e8-b74b-4bc1-9b93-8c785dae7c0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:49.953240 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a761e8-b74b-4bc1-9b93-8c785dae7c0e service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:49.953713 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a761e8-b74b-4bc1-9b93-8c785dae7c0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:49.954306 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a761e8-b74b-4bc1-9b93-8c785dae7c0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:49.957227 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f8a761e8-b74b-4bc1-9b93-8c785dae7c0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:49.974681 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a761e8-b74b-4bc1-9b93-8c785dae7c0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:49.982122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a761e8-b74b-4bc1-9b93-8c785dae7c0e service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:50.017048 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1433/2881] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:33:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:53.447770 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b9892-7058-4201-9486-57c1b9e25d87 None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:53.448708 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b9892-7058-4201-9486-57c1b9e25d87 None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:53.448991 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b9892-7058-4201-9486-57c1b9e25d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:53.473291 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-c91b9892-7058-4201-9486-57c1b9e25d87 None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:53.474925 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c91b9892-7058-4201-9486-57c1b9e25d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:53.486026 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1449/2882] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 15:33:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:53.501358 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a761e8-b74b-4bc1-9b93-8c785dae7c0e service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:53.516223 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a761e8-b74b-4bc1-9b93-8c785dae7c0e service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:53.530861 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c483b-8b98-42c8-8026-9276aaed3368 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:53.532777 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c483b-8b98-42c8-8026-9276aaed3368 service swift] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:53.533549 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c483b-8b98-42c8-8026-9276aaed3368 service swift] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:53.534439 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c483b-8b98-42c8-8026-9276aaed3368 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:53.551498 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c483b-8b98-42c8-8026-9276aaed3368 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:53.555221 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c483b-8b98-42c8-8026-9276aaed3368 service swift] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:53.571390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1434/2883] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:33:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:33:55.771525 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b9892-7058-4201-9486-57c1b9e25d87 None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:55.773302 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c91b9892-7058-4201-9486-57c1b9e25d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:55.799739 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b9892-7058-4201-9486-57c1b9e25d87 None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:55.807835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70e7f2-1bfc-449f-9ab8-6f7ac7e0883a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:55.807835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70e7f2-1bfc-449f-9ab8-6f7ac7e0883a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:55.807835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70e7f2-1bfc-449f-9ab8-6f7ac7e0883a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:55.807835 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70e7f2-1bfc-449f-9ab8-6f7ac7e0883a admin admin] PATH_INFO: `/v3/users/9d3652448ec141918d94617077b9ce81` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:55.809118 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70e7f2-1bfc-449f-9ab8-6f7ac7e0883a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d3652448ec141918d94617077b9ce81)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:55.816219 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70e7f2-1bfc-449f-9ab8-6f7ac7e0883a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:55.842148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:55.883443 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ad70e7f2-1bfc-449f-9ab8-6f7ac7e0883a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d3652448ec141918d94617077b9ce81'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:55.892747 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ad70e7f2-1bfc-449f-9ab8-6f7ac7e0883a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d3652448ec141918d94617077b9ce81'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:55.893367 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ad70e7f2-1bfc-449f-9ab8-6f7ac7e0883a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d3652448ec141918d94617077b9ce81'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:55.902653 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ad70e7f2-1bfc-449f-9ab8-6f7ac7e0883a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d3652448ec141918d94617077b9ce81'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:55.916497 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-ad70e7f2-1bfc-449f-9ab8-6f7ac7e0883a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d3652448ec141918d94617077b9ce81'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:55.923184 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1450/2884] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:33:55 2022] DELETE /identity/v3/users/9d3652448ec141918d94617077b9ce81 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:56.123665 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c483b-8b98-42c8-8026-9276aaed3368 service swift] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:56.140255 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c483b-8b98-42c8-8026-9276aaed3368 service swift] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:56.205398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074695e-6916-4beb-bff0-c80f56ff07d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:56.206119 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074695e-6916-4beb-bff0-c80f56ff07d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:56.206398 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074695e-6916-4beb-bff0-c80f56ff07d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:56.206674 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074695e-6916-4beb-bff0-c80f56ff07d7 admin admin] PATH_INFO: `/v3/projects/1947572d5cd646d08f4efaa2ef17e271` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:56.207880 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074695e-6916-4beb-bff0-c80f56ff07d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1947572d5cd646d08f4efaa2ef17e271)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:56.210734 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074695e-6916-4beb-bff0-c80f56ff07d7 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:56.264081 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b074695e-6916-4beb-bff0-c80f56ff07d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1947572d5cd646d08f4efaa2ef17e271', 'request_id': 'req-b074695e-6916-4beb-bff0-c80f56ff07d7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:56.270099 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:56.271723 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b074695e-6916-4beb-bff0-c80f56ff07d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1947572d5cd646d08f4efaa2ef17e271', 'request_id': 'req-b074695e-6916-4beb-bff0-c80f56ff07d7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:56.282663 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b074695e-6916-4beb-bff0-c80f56ff07d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1947572d5cd646d08f4efaa2ef17e271', 'request_id': 'req-b074695e-6916-4beb-bff0-c80f56ff07d7'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:56.295087 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b074695e-6916-4beb-bff0-c80f56ff07d7 admin admin] The token cache is being invalidate because project 1947572d5cd646d08f4efaa2ef17e271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:33:56.295603 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b074695e-6916-4beb-bff0-c80f56ff07d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:56.296346 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-b074695e-6916-4beb-bff0-c80f56ff07d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:56.298150 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1435/2885] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:33:56 2022] DELETE /identity/v3/projects/1947572d5cd646d08f4efaa2ef17e271 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:58.455376 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70e7f2-1bfc-449f-9ab8-6f7ac7e0883a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:58.456878 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-ad70e7f2-1bfc-449f-9ab8-6f7ac7e0883a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:58.470981 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70e7f2-1bfc-449f-9ab8-6f7ac7e0883a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:58.532154 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55295191-245f-4ec7-adde-334dd131aaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:58.532798 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55295191-245f-4ec7-adde-334dd131aaa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:58.533072 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55295191-245f-4ec7-adde-334dd131aaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:58.533332 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55295191-245f-4ec7-adde-334dd131aaa6 admin admin] PATH_INFO: `/v3/users/0898fda8be3743f093dbb216f21eb931` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:58.534550 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55295191-245f-4ec7-adde-334dd131aaa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0898fda8be3743f093dbb216f21eb931)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:58.536491 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55295191-245f-4ec7-adde-334dd131aaa6 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:58.600469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-55295191-245f-4ec7-adde-334dd131aaa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0898fda8be3743f093dbb216f21eb931'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:58.609565 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-55295191-245f-4ec7-adde-334dd131aaa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0898fda8be3743f093dbb216f21eb931'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:58.610270 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-55295191-245f-4ec7-adde-334dd131aaa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0898fda8be3743f093dbb216f21eb931'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:58.618510 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-55295191-245f-4ec7-adde-334dd131aaa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0898fda8be3743f093dbb216f21eb931'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:58.633306 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-55295191-245f-4ec7-adde-334dd131aaa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0898fda8be3743f093dbb216f21eb931'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:58.640758 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1451/2886] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:33:58 2022] DELETE /identity/v3/users/0898fda8be3743f093dbb216f21eb931 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:58.845172 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074695e-6916-4beb-bff0-c80f56ff07d7 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:58.847242 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-b074695e-6916-4beb-bff0-c80f56ff07d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:58.869804 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074695e-6916-4beb-bff0-c80f56ff07d7 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:58.945044 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230f1fd-a67e-4a38-be9c-e6dddce98521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:58.945697 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230f1fd-a67e-4a38-be9c-e6dddce98521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:58.946009 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230f1fd-a67e-4a38-be9c-e6dddce98521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:58.946286 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230f1fd-a67e-4a38-be9c-e6dddce98521 admin admin] PATH_INFO: `/v3/projects/97f413eea3b945cc843040b11d501993` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:58.947474 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3230f1fd-a67e-4a38-be9c-e6dddce98521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97f413eea3b945cc843040b11d501993)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:58.949466 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3230f1fd-a67e-4a38-be9c-e6dddce98521 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:58.998309 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3230f1fd-a67e-4a38-be9c-e6dddce98521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97f413eea3b945cc843040b11d501993', 'request_id': 'req-3230f1fd-a67e-4a38-be9c-e6dddce98521'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.004003 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3230f1fd-a67e-4a38-be9c-e6dddce98521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97f413eea3b945cc843040b11d501993', 'request_id': 'req-3230f1fd-a67e-4a38-be9c-e6dddce98521'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.012565 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3230f1fd-a67e-4a38-be9c-e6dddce98521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97f413eea3b945cc843040b11d501993', 'request_id': 'req-3230f1fd-a67e-4a38-be9c-e6dddce98521'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.024872 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3230f1fd-a67e-4a38-be9c-e6dddce98521 admin admin] The token cache is being invalidate because project 97f413eea3b945cc843040b11d501993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:33:59.025175 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3230f1fd-a67e-4a38-be9c-e6dddce98521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.025853 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-3230f1fd-a67e-4a38-be9c-e6dddce98521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.027402 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1436/2887] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:33:58 2022] DELETE /identity/v3/projects/97f413eea3b945cc843040b11d501993 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:59.031789 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55295191-245f-4ec7-adde-334dd131aaa6 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:59.033582 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-55295191-245f-4ec7-adde-334dd131aaa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:59.048820 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55295191-245f-4ec7-adde-334dd131aaa6 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:59.058056 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:59.112152 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72083cc-2e5e-47a7-83ff-b4392790e86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:59.112698 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72083cc-2e5e-47a7-83ff-b4392790e86a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:59.112978 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72083cc-2e5e-47a7-83ff-b4392790e86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:59.113246 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72083cc-2e5e-47a7-83ff-b4392790e86a admin admin] PATH_INFO: `/v3/users/452f003eb205486895eb397d5f008d09` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:59.114624 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72083cc-2e5e-47a7-83ff-b4392790e86a admin admin] RBAC: Authorizing `identity:delete_user(user_id=452f003eb205486895eb397d5f008d09)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:59.116545 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72083cc-2e5e-47a7-83ff-b4392790e86a admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:59.178944 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c72083cc-2e5e-47a7-83ff-b4392790e86a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '452f003eb205486895eb397d5f008d09'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.188671 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c72083cc-2e5e-47a7-83ff-b4392790e86a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '452f003eb205486895eb397d5f008d09'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.189402 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c72083cc-2e5e-47a7-83ff-b4392790e86a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '452f003eb205486895eb397d5f008d09'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.196454 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c72083cc-2e5e-47a7-83ff-b4392790e86a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '452f003eb205486895eb397d5f008d09'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.210905 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-c72083cc-2e5e-47a7-83ff-b4392790e86a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '452f003eb205486895eb397d5f008d09'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.217425 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1452/2888] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:33:59 2022] DELETE /identity/v3/users/452f003eb205486895eb397d5f008d09 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:59.419922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230f1fd-a67e-4a38-be9c-e6dddce98521 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:59.421981 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-3230f1fd-a67e-4a38-be9c-e6dddce98521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:59.436720 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230f1fd-a67e-4a38-be9c-e6dddce98521 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:59.499461 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:59.500111 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:59.500387 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:59.500663 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb admin admin] PATH_INFO: `/v3/projects/f3c0f7029fda4d69b7d5bfd20401e79d` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:59.502235 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3c0f7029fda4d69b7d5bfd20401e79d)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:59.505355 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:59.525609 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:59.573413 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3c0f7029fda4d69b7d5bfd20401e79d', 'request_id': 'req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.581618 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3c0f7029fda4d69b7d5bfd20401e79d', 'request_id': 'req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.591067 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3c0f7029fda4d69b7d5bfd20401e79d', 'request_id': 'req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.604180 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb admin admin] The token cache is being invalidate because project f3c0f7029fda4d69b7d5bfd20401e79d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:33:59.604466 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.605182 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:59.606843 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1437/2889] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:33:59 2022] DELETE /identity/v3/projects/f3c0f7029fda4d69b7d5bfd20401e79d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:09.214337 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=47,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:09.600922 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=17,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:52.251739 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72083cc-2e5e-47a7-83ff-b4392790e86a admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:52.253916 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-c72083cc-2e5e-47a7-83ff-b4392790e86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:52.383815 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72083cc-2e5e-47a7-83ff-b4392790e86a admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:52.495393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f01100-39f5-429b-b8ab-c31b5da17fd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de0ff99a2b74979a28a8b10e9ebdfcd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:52.495393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f01100-39f5-429b-b8ab-c31b5da17fd4 service nova] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:52.495393 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f01100-39f5-429b-b8ab-c31b5da17fd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:52.496305 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f01100-39f5-429b-b8ab-c31b5da17fd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:52.498054 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-89f01100-39f5-429b-b8ab-c31b5da17fd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:52.513162 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f01100-39f5-429b-b8ab-c31b5da17fd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:52.515353 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f01100-39f5-429b-b8ab-c31b5da17fd4 service nova] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:52.534075 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1453/2890] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:52.667800 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:52.671450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:52.774194 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770e41c-36fe-4d6a-a3b2-db4a981bc6eb admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:52.778390 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981fdf4-da2f-46f1-a158-81c3878459ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ef6577b10f7426a925d594e627f27cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:52.779191 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981fdf4-da2f-46f1-a158-81c3878459ea service neutron] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:52.779480 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981fdf4-da2f-46f1-a158-81c3878459ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:52.779755 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981fdf4-da2f-46f1-a158-81c3878459ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:52.781923 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4981fdf4-da2f-46f1-a158-81c3878459ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:52.793647 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981fdf4-da2f-46f1-a158-81c3878459ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:52.795594 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981fdf4-da2f-46f1-a158-81c3878459ea service neutron] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:52.806634 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1438/2891] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:34:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:35:01.195469 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf1b64-b64a-42a5-9caf-658e1fb6353a None None] REQUEST_METHOD: `POST` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:01.196518 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf1b64-b64a-42a5-9caf-658e1fb6353a None None] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:01.196699 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf1b64-b64a-42a5-9caf-658e1fb6353a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:01.278646 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.auth.core [None req-6ddf1b64-b64a-42a5-9caf-658e1fb6353a None None] MFA Rules not processed for user `38b18e5d1e8843c6a08b1f5609cc60d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80292) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:35:01.280499 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6ddf1b64-b64a-42a5-9caf-658e1fb6353a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:01.378197 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=100 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:01.390275 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1454/2892] 10.222.0.48 () {58 vars in 967 bytes} [Mon Aug 1 15:35:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:35:01.462790 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981fdf4-da2f-46f1-a158-81c3878459ea service neutron] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:01.464497 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-4981fdf4-da2f-46f1-a158-81c3878459ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:01.600281 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981fdf4-da2f-46f1-a158-81c3878459ea service neutron] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:01.606869 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ea36d-ec02-4ab1-a7c3-b7ef71c051ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a26f54accf4264aef646c3757de7e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:01.607970 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ea36d-ec02-4ab1-a7c3-b7ef71c051ac service swift] REQUEST_METHOD: `GET` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:01.608359 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ea36d-ec02-4ab1-a7c3-b7ef71c051ac service swift] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:01.608735 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ea36d-ec02-4ab1-a7c3-b7ef71c051ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:01.620067 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ea36d-ec02-4ab1-a7c3-b7ef71c051ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:01.622810 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ea36d-ec02-4ab1-a7c3-b7ef71c051ac service swift] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:01.633204 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1439/2893] 10.222.0.48 () {60 vars in 1384 bytes} [Mon Aug 1 15:35:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:35:10.615513 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf1b64-b64a-42a5-9caf-658e1fb6353a None None] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:10.617890 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-6ddf1b64-b64a-42a5-9caf-658e1fb6353a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:10.706158 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf1b64-b64a-42a5-9caf-658e1fb6353a None None] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:10.711631 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6485a4-2bdb-4fd1-bf13-2f9505c13414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:10.712319 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6485a4-2bdb-4fd1-bf13-2f9505c13414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:10.712587 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6485a4-2bdb-4fd1-bf13-2f9505c13414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:10.712856 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6485a4-2bdb-4fd1-bf13-2f9505c13414 admin admin] PATH_INFO: `/v3/users/5f2b4bba3ed5400d8ab618a6ccbc04d4` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:10.714096 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6485a4-2bdb-4fd1-bf13-2f9505c13414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f2b4bba3ed5400d8ab618a6ccbc04d4)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:10.716099 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6485a4-2bdb-4fd1-bf13-2f9505c13414 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:10.780213 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-db6485a4-2bdb-4fd1-bf13-2f9505c13414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f2b4bba3ed5400d8ab618a6ccbc04d4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:10.790509 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-db6485a4-2bdb-4fd1-bf13-2f9505c13414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f2b4bba3ed5400d8ab618a6ccbc04d4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:10.791249 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-db6485a4-2bdb-4fd1-bf13-2f9505c13414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f2b4bba3ed5400d8ab618a6ccbc04d4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:10.802627 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-db6485a4-2bdb-4fd1-bf13-2f9505c13414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f2b4bba3ed5400d8ab618a6ccbc04d4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:10.819123 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-db6485a4-2bdb-4fd1-bf13-2f9505c13414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f2b4bba3ed5400d8ab618a6ccbc04d4'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:10.826428 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1455/2894] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:35:10 2022] DELETE /identity/v3/users/5f2b4bba3ed5400d8ab618a6ccbc04d4 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:35:11.081704 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ea36d-ec02-4ab1-a7c3-b7ef71c051ac service swift] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:11.104372 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ea36d-ec02-4ab1-a7c3-b7ef71c051ac service swift] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:11.184075 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f72031-8296-4aa0-900f-61355bea4f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:11.185112 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f72031-8296-4aa0-900f-61355bea4f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:11.185518 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f72031-8296-4aa0-900f-61355bea4f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:11.186071 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f72031-8296-4aa0-900f-61355bea4f52 admin admin] PATH_INFO: `/v3/projects/ff70f42c96804182af2e61e501ee8a21` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:11.188202 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f72031-8296-4aa0-900f-61355bea4f52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff70f42c96804182af2e61e501ee8a21)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:11.191622 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f72031-8296-4aa0-900f-61355bea4f52 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:11.220450 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=100 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:11.274653 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-39f72031-8296-4aa0-900f-61355bea4f52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff70f42c96804182af2e61e501ee8a21', 'request_id': 'req-39f72031-8296-4aa0-900f-61355bea4f52'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:11.283234 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-39f72031-8296-4aa0-900f-61355bea4f52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff70f42c96804182af2e61e501ee8a21', 'request_id': 'req-39f72031-8296-4aa0-900f-61355bea4f52'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:11.296250 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-39f72031-8296-4aa0-900f-61355bea4f52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff70f42c96804182af2e61e501ee8a21', 'request_id': 'req-39f72031-8296-4aa0-900f-61355bea4f52'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:11.315847 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-39f72031-8296-4aa0-900f-61355bea4f52 admin admin] The token cache is being invalidate because project ff70f42c96804182af2e61e501ee8a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:35:11.316317 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-39f72031-8296-4aa0-900f-61355bea4f52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:11.317150 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-39f72031-8296-4aa0-900f-61355bea4f52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:11.319006 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1440/2895] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:35:11 2022] DELETE /identity/v3/projects/ff70f42c96804182af2e61e501ee8a21 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:11.324638 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6485a4-2bdb-4fd1-bf13-2f9505c13414 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:11.327456 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-db6485a4-2bdb-4fd1-bf13-2f9505c13414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:11.351441 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6485a4-2bdb-4fd1-bf13-2f9505c13414 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:11.435933 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11795c-c5cb-4ba8-a38a-49a806a852f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:11.436588 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11795c-c5cb-4ba8-a38a-49a806a852f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:11.436866 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11795c-c5cb-4ba8-a38a-49a806a852f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:11.437139 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11795c-c5cb-4ba8-a38a-49a806a852f3 admin admin] PATH_INFO: `/v3/users/e97c98ea43e9441d9e0ba3051410a577` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:11.438592 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11795c-c5cb-4ba8-a38a-49a806a852f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e97c98ea43e9441d9e0ba3051410a577)` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:11.440553 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11795c-c5cb-4ba8-a38a-49a806a852f3 admin admin] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:11.457935 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:11.505576 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-9b11795c-c5cb-4ba8-a38a-49a806a852f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e97c98ea43e9441d9e0ba3051410a577'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:11.514360 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-9b11795c-c5cb-4ba8-a38a-49a806a852f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e97c98ea43e9441d9e0ba3051410a577'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:11.515001 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-9b11795c-c5cb-4ba8-a38a-49a806a852f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e97c98ea43e9441d9e0ba3051410a577'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:11.524148 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-9b11795c-c5cb-4ba8-a38a-49a806a852f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e97c98ea43e9441d9e0ba3051410a577'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:11.546210 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.notifications [None req-9b11795c-c5cb-4ba8-a38a-49a806a852f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e97c98ea43e9441d9e0ba3051410a577'} {{(pid=80292) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:11.556002 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1456/2896] 10.222.0.48 () {58 vars in 1224 bytes} [Mon Aug 1 15:35:11 2022] DELETE /identity/v3/users/e97c98ea43e9441d9e0ba3051410a577 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:11.798119 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f72031-8296-4aa0-900f-61355bea4f52 admin admin] Authenticating user token {{(pid=80291) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:35:11.800361 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.fernet_utils [None req-39f72031-8296-4aa0-900f-61355bea4f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80291) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:35:11.833996 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f72031-8296-4aa0-900f-61355bea4f52 admin admin] Validating token access rules against request {{(pid=80291) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:35:11.932122 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a52da-01b6-4b1d-9ce1-57c5d1ee6a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b4b6acdd88e421a86924fd2c155794f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef3b9d837c2248a7ae12fa99e632e776', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80291) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:35:11.932826 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a52da-01b6-4b1d-9ce1-57c5d1ee6a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:35:11.933138 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a52da-01b6-4b1d-9ce1-57c5d1ee6a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:35:11.933448 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a52da-01b6-4b1d-9ce1-57c5d1ee6a97 admin admin] PATH_INFO: `/v3/projects/a5317b74e40d4a8fa5b0cc2beb9e005c` {{(pid=80291) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:35:11.934801 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a52da-01b6-4b1d-9ce1-57c5d1ee6a97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5317b74e40d4a8fa5b0cc2beb9e005c)` {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:35:11.936860 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a52da-01b6-4b1d-9ce1-57c5d1ee6a97 admin admin] RBAC: Authorization granted {{(pid=80291) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:35:12.011336 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d87a52da-01b6-4b1d-9ce1-57c5d1ee6a97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5317b74e40d4a8fa5b0cc2beb9e005c', 'request_id': 'req-d87a52da-01b6-4b1d-9ce1-57c5d1ee6a97'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:12.017769 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d87a52da-01b6-4b1d-9ce1-57c5d1ee6a97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5317b74e40d4a8fa5b0cc2beb9e005c', 'request_id': 'req-d87a52da-01b6-4b1d-9ce1-57c5d1ee6a97'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:12.027551 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d87a52da-01b6-4b1d-9ce1-57c5d1ee6a97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5317b74e40d4a8fa5b0cc2beb9e005c', 'request_id': 'req-d87a52da-01b6-4b1d-9ce1-57c5d1ee6a97'} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:12.051524 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d87a52da-01b6-4b1d-9ce1-57c5d1ee6a97 admin admin] The token cache is being invalidate because project a5317b74e40d4a8fa5b0cc2beb9e005c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80291) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:35:12.051912 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d87a52da-01b6-4b1d-9ce1-57c5d1ee6a97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:12.052763 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG keystone.notifications [None req-d87a52da-01b6-4b1d-9ce1-57c5d1ee6a97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80291) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:35:12.055265 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: [pid: 80291|app: 0|req: 1441/2897] 10.222.0.48 () {58 vars in 1233 bytes} [Mon Aug 1 15:35:11 2022] DELETE /identity/v3/projects/a5317b74e40d4a8fa5b0cc2beb9e005c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:21.549900 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:DELETE=9,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:22.045029 dv-cinder-847132-3-iscsi devstack@keystone.service[80291]: DEBUG dbcounter [-] [80291] Writing DB stats keystone:SELECT=69,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80291) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:36:59.454559 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11795c-c5cb-4ba8-a38a-49a806a852f3 admin admin] Authenticating user token {{(pid=80292) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:36:59.456021 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-9b11795c-c5cb-4ba8-a38a-49a806a852f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:59.536155 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11795c-c5cb-4ba8-a38a-49a806a852f3 admin admin] Validating token access rules against request {{(pid=80292) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:36:59.599333 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45fb42-d3b2-4ae3-ac25-ef3951aa881b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '098d8d7e35804ccc931ddfba07f688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '218a9ee5cb2842248b6995d2562bb74b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80292) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:36:59.599975 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45fb42-d3b2-4ae3-ac25-ef3951aa881b service placement] REQUEST_METHOD: `GET` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:36:59.600254 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45fb42-d3b2-4ae3-ac25-ef3951aa881b service placement] SCRIPT_NAME: `/identity` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:36:59.600525 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45fb42-d3b2-4ae3-ac25-ef3951aa881b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80292) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:36:59.602491 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.fernet_utils [None req-8b45fb42-d3b2-4ae3-ac25-ef3951aa881b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80292) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:36:59.620764 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45fb42-d3b2-4ae3-ac25-ef3951aa881b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:36:59.622488 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45fb42-d3b2-4ae3-ac25-ef3951aa881b service placement] RBAC: Authorization granted {{(pid=80292) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:36:59.635231 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: [pid: 80292|app: 0|req: 1457/2898] 10.222.0.48 () {60 vars in 1365 bytes} [Mon Aug 1 15:36:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:37:09.628711 dv-cinder-847132-3-iscsi devstack@keystone.service[80292]: DEBUG dbcounter [-] [80292] Writing DB stats keystone:SELECT=58 {{(pid=80292) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}